Friday, July 10, 2026

Google apps script issue - script.google.com refused to connect - google glitch

One of our servers which had a google apps script embedded in a php page started showing "script.google.com refused to connect" - the issue was noticed last afternoon. We checked that no changes had been made to the script, we did have
.setXFrameOptionsMode(HtmlService.XFrameOptionsMode.ALLOWALL);
as mentioned in

The only other change was to install and set up fail2ban on the server, that was unlikely to be the cause. 

"If no changes have been made to the script, then perhaps it is a transient google problem, and might be resolved in a few hours.

If not, we would need to check if google apps scripts have any recent policy changes."

It might have been a google glitch, since last night the issue was resolved by itself. Looked like the outage was for around 8-10 hours.

Tuesday, July 07, 2026

reminder to take a break - Mac version

I wanted to recreate the earlier "reminder to take a break" cron job script on MacOS. Asked Claude about it, and it suggested using launchd instead of cron, since cron jobs run without access to the display by default. After several rounds of trial and error, here is the method that worked.

nano ~/Library/LaunchAgents/com.yourname.walkreminder.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.yourname.walkreminder</string>

    <key>ProgramArguments</key>
    <array>
        <string>/bin/bash</string>
        <string>-c</string>
        <string>osascript -e 'display notification "Time to take a walk" with title "Reminder" sound name "Ping"'; afplay /Users/yourusername/Sounds/Walk.mp3</string>
    </array>

    <key>StartCalendarInterval</key>
    <array>
        <dict>
            <key>Hour</key>
            <integer>14</integer>
            <key>Minute</key>
            <integer>0</integer>
        </dict>
    </array>

    <key>StandardOutPath</key>
    <string>/tmp/walkreminder.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/walkreminder.err</string>
</dict>
</plist>

In the above, the schedule is to run once a day at 14:00 hours. For running every half an hour from the time the user was logged on, we can use 

<key>StartInterval</key>
  <integer>1800</integer>

instead of <key>StartCalendarInterval</key>

Or, in my case, since I wanted every half an hour aligned to 0:00 and 0:30 exactly, 

 <key>StartCalendarInterval</key>
    <array>
        <dict>
            <key>Minute</key>
            <integer>0</integer>
        </dict> 
        <dict>
            <key>Minute</key>
            <integer>30</integer>
        </dict>
    </array>

We had to use a Sounds directory which we had created, since afplay could not access the Downloads directory when run from launchd.

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.yourname.walkreminder.plist

and test with

launchctl kickstart -k gui/$(id -u)/com.yourname.walkreminder

Troubleshooting can be done with
plutil -lint ~/Library/LaunchAgents/com.yourname.walkreminder.plist
launchctl list | grep walkreminder
cat /tmp/walkreminder.err


Friday, July 03, 2026

incident and response - malicious bots

One of our servers, which hosted a dot net backend as well as some wordpress sites, showed some outages two days in a row, restored by restarting. Feeding the apache access logs to Gemini via aistudio.google.com, Gemini said,

The logs clearly show an Apache HTTP
Server running PHP and hosting a WordPress application.

The logs indicate that your server was hit by an aggressive automated attack
(likely a botnet) starting around 07:33:35, which appears to have either
succeeded in exploiting a vulnerability or overloaded the server, causing it to
crash shortly after.

(Server had run out of memory). 

Gemini suggested the following - 

  1. Hide and Protect Your Origin IP - which we may not do
  2. Patch and Update the Server Software - which we're doing
  3. Implement a Web Application Firewall (WAF) - which is in place
  4. Clean Up WordPress Configuration - "Open your wp-config.php file, locate line 63, and remove the    duplicate definition of WP_AUTO_UPDATE_CORE. While this didn't cause the crash, it eats up server I/O and makes reading logs difficult."
  5. Restrict PHP Execution in Uploads Directories
  6. Implement Intrusion Prevention (Fail2Ban)

Then, feeding the access logs of the dot net api to Gemini 
and as suggested by Gemini,
journalctl -u ourapi.service --since "2026-06-28 07:15:00" --until "2026-06-28 07:45:00"

Gemini gave some recommendations like returning 0 instead of a 500 error for no data found -
"As we saw in the access logs, when the mobile app sees
    this 500 error, its poorly designed error-handling logic says: "Something
    went wrong! Retry the entire sync process!" It then proceeds to download 4MB
    of lesson data, hits the notification endpoint again, gets another 500
    error, and repeats the cycle every 15 seconds until your server runs out of
    memory and crashes."
- this may or may not be actually what is happening here, since Gemini just saw the logs and does not have access to the code.

In any case, I've enabled bot protection on Cloudflare for this domain, and also installed and configured Fail2ban.

On Cloudflare, 
ourdomain.org > Security > Security rules > Custom rules

Block .env scans
URI Path equals .env
Block

 also added Cloudflare's default rate limiting rule,

Leaked credential check [Template]
Password Leaked equals true
Block

also enabled Cloudflare's AI blocking tools - AI Labyrinth enabled, and Block AI training bots on all pages.

Then, installed and set up Fail2ban on the server, monitoring the apache web server and sshd logs. It will temporarily block ip addresses which fail authentication repeatedly, or which repeatedly request non-existent files like malicious bots.

Gemini via aistudio.google.com gave step-by-step instructions for installation and setup of Fail2ban. 

sudo apt install fail2ban -y
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

(we can optionally edit these,
bantime  = 1h
findtime = 10m
maxretry = 5
)

Enable the Apache jails with
sudo nano /etc/fail2ban/jail.local
[apache-auth]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

In case the logpath is custom - fail2ban looks by default for
Error logs: /var/log/apache2/*error.log
Access logs: /var/log/apache2/*access.log

In the case of one of our servers, we needed to change this to
logpath  = %(apache_error_log)s
           /var/www/mysite/logs/custom-error*.log

More Apache jails to enable - basically need to add the enabled = true line - 

[apache-badbots]
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
bantime  = 48h
maxretry = 1

[apache-noscript]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-overflows]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-nohome]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-botsearch]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

Then we can test

sudo systemctl start fail2ban
sudo systemctl enable fail2ban
sudo systemctl status fail2ban

and check the enabled jails with

sudo fail2ban-client status

On one server, we had to resolve
ERROR   Failed during configuration: Have not found any log file
for sshd jail

For that, we had to change the block to
[sshd]
enabled = true
port    = ssh
backend = systemd

since that server was using systemd, and traditional text log files like /var/log/auth.log (which Fail2Ban looks for by default for SSH) are no longer created. After the change, after restarting the service, we can check that jail alone with

sudo fail2ban-client status sshd

Then, for using a different port rather than port 22 for sshd - 

[sshd]
enabled = true
port    = 2022
backend = systemd
(if port 2022 is being used) 
and so on, because that is the port which fail2ban would ban using the machine's firewall, iptables or nftables as the case may be.

To see the rules, we can use
sudo nft list ruleset # for nftables, Ubuntu 24.04
sudo iptables -nL # for iptables, earlier Ubuntu etc



Tuesday, June 23, 2026

SIR online form submission - Election Commission of India

Currently, a "Special Intensive Revision" or SIR is going on for the elector rolls in many parts of India. My colleague told me about the process going on in the local administration's office, and told me that I could submit the form online also.

An internet search revealed

Luckily, my mobile phone number was already available in the system, so when I clicked on sign-up, it indicated that my number is already registered. Then tried log in - an OTP was sent to my phone, and I could log in. 

The process which I followed was
  • Using the search function, found my name in the 2002 electoral rolls - it was on page 2 or 3, I skimmed through the results based on polling station name (we need to be able to read Telugu, the local language, for doing this.) I saved the info listed there - Assembly constituency, polling station and  
  • Next, went to the home page and clicked on "Fill Enumeration form"
  • We have to go through several OTPs, confirm our details, ensure that the name displayed is the same name as on Aadhaar - otherwise we need to do the form submission manually, details of the BLO (Booth Level Officer) are displayed.
  • Then the form opens up, where we need to fill up details and a recent photograph. The initial photographs I uploaded were not recognized "no face was recognized in this photograph" or something like that, after a wait of a minute or two. Thinking that this might be due to spectacles in the picture, clicked a new photo of myself without spectacles, and tried the upload.
  • Yesterday at around 2.30 pm, the form came up for submission after that, though the photo was not visible in the preview. But when I clicked on submit, the "Aadhaar OTP" method of verification did not succeed, it said wrong OTP every time.
  • This morning at around 8.50 am, went through the same process, this time the "Aadhaar OTP" method of verification was successful - again had to wait for a few minutes after uploading the photo in order to get the submit form enabled - and I got a downloadable receipt of the form submitted, with the new polling station number, constituency name and so on.
  • Found that I could also download my voter-id card (EPIC - Election Photo Identity Card) - from https://voters.eci.gov.in/ 

Thursday, June 18, 2026

rewrite of restic backup script and adding email alert

Found that our restic backup script had stopped running from Apr 20th - the log files were dated as last modified on that date. This could have been due to 
(a) an update overwriting the restic binary with an older version of ubuntu's restic via apt, which did not work, giving "backend not supported" errors for Azure Storage
(b) an additional issue that our Azure Storage account had become unavailable at some point of time due to a lapsed subscription 

So, created a new Storage account in an active subscription in the same resource group, which defaulted to the same Central India region for the storage account as we wanted. Used the same name as the earlier script's storage account name, and created a Blob container also with the same name as used by the script. 

Due to the point (a) above, this was not sufficient to make the script work, even for it to initialize the storage with lines like
restic -r azure:our-data-bk:/ourdata init
Fatal: create repository at azure:our-data-bk:/ourdata failed: invalid backend

As suggested by Gemini, the solution was to uninstall restic via apt, and to then download the latest restic and copy it to /usr/local/bin

wget https://github.com/restic/restic/releases/download/v0.18.1/restic_0.18.1_linux_amd64.bz2
bzip2 -d restic_0.18.1_linux_amd64.bz2
chmod +x restic_0.18.1_linux_amd64
sudo mv restic_0.18.1_linux_amd64 /usr/local/bin/restic

sudo apt remove restic

Then,
restic version
-bash: /usr/bin/restic: No such file or directory

We needed to tell bash to refresh the location - 
hash -r

Then the restic version showed the correct version, and the init also was successful. 

I wanted to get emails on future failures instead of the script failing silently, so Gemini helped with this script, which I have modified to change the passwords etc

#!/bin/bash
#This will run Restic backups from cron.
export RESTIC_PASSWORD=ourpw
export AZURE_ACCOUNT_NAME=ourname
#export AZURE_ACCOUNT_SAS="sv=2022-11-02&ss=bfqt&srt=c&sp=not_used_this_time%3D"
export AZURE_ACCOUNT_KEY="mVthisisthekey99999wPlEA=="
# not using rclone+gdrive due to slow,timeouts,rate-limits
#RCLONE_CONFIG=/home/user/.config/rclone/rclone.conf
#create new repo
# https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage
#restic -r azure:our-data-bk:/ourdata init
# take backups
/usr/local/bin/restic  -r azure:sssvv-data-bk:/1data  --verbose backup  /var/www/1_data_disk/1_data/filedir > /home/user/1resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/2data  --verbose backup  /var/www/1_data_disk/2_data/filedir > /home/user/2resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/3data  --verbose backup  /var/www1_data_disk/3_data/filedir > /home/user/3resticlog.txt 2>&1
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
    SUBJECT="SUCCESS: Weekly Restic Backup"
    MESSAGE="Your weekly restic backup completed successfully."
else
    SUBJECT="ALERT: Weekly Restic Backup FAILED"
    MESSAGE="WARNING: Your restic backup FAILED with exit code $EXIT_CODE. Please investigate immediately."
fi
mail -s "$SUBJECT" "my@email.org" << EOF
$MESSAGE

Here is the log output:
--------------------------------------------------
$(cat "/home/user/3resticlog.txt")
EOF



Wednesday, June 17, 2026

Updating expiring Azure Linux Virtual Machine Secure Boot 2011 certificates

Microsoft Azure's email notification asked us to update the secure boot certificates before the end of the month, and pointed us to verification, and if necessary updating, steps. The "vendor recommended" documentation for Ubuntu support was a bit contradictory - saying that rollout had been paused - so took the help of ChatGPT and Gemini for completing the process. First took up a non-critical VM, completed that, and then went on to the others.

sudo snap install fwupd
sudo fwupdmgr refresh
sudo fwupdmgr update
#(say yes, yes, and yes to reboot)

Gemini reassured that the devices listed with "no updates" are not a concern, we should only check whether the mokutil tests below work OK.

As per the verification link above, 
Tested with
mokutil --db | grep "2023"
            Not Before: Jun 13 19:21:47 2023 GMT
        Subject: C=US, O=Microsoft Corporation, CN=Microsoft UEFI CA 2023
mokutil --kek | grep "2023"
            Not Before: Mar  2 20:21:35 2023 GMT
        Subject: C=US, O=Microsoft Corporation, CN=Microsoft Corporation KEK 2K CA 2023

Removed the fwupd snap, and also removed snap itself to prevent bloat
sudo snap remove fwupd
(and remove snap itself on ELS)
snap list
(if nothing other than core, core20, lxd, or snapd, can remove)

sudo systemctl disable --now snapd.service snapd.socket
sudo apt-get purge -y snapd
sudo rm -rf /snap /var/snap /var/lib/snapd /var/cache/snapd /usr/lib/snapd

Found that the L VM was already up-to-date since it was a newer VM, created in 2025.

SDev2 had to be updated in the same manner as for the HAPROXY VM above.

SSS web server also had to be updated in the same manner.

On the AWS VM, I see

mokutil --sb-state
EFI variables are not supported on this system

Gemini says,

You do not need to do anything for this AWS VM. You are completely in the clear.
Seeing EFI variables are not supported on this system means that this specific EC2 instance is not using UEFI Secure Boot at all. In fact, it is likely booting using Legacy BIOS rather than UEFI.


With that, all the VMs seem to be accounted for.

Tuesday, June 16, 2026

Samsung phone Gboard voice typing icon vanished

At some point of time, after some upgrades / updates, the voice-typing mic icon vanished, for the Gboard keyboard on my Samsung M34 5G phone. 

There were lots of contradictory (and perhaps out-dated) info online on how to re-enable voice-typing, but what worked for me was as follows. Gboard was already set as the default keyboard, and not Samsung keyboard in the settings. 

When Gboard was visible, the path to set this was,
the icon to show more items - 
then the Gboard settings icon,

 followed by voice-typing

and finally slide to enable "Use voice typing" as below.

Then the mic icon at the top right corner of Gboard becomes visible.


Sunday, June 07, 2026

upgrade mesa on Ubuntu 22

For testing OpenSpace on Ubuntu 22, I wanted to upgrade the Mesa provided OpenGL to v4.6. According to https://linuxcapable.com/how-to-upgrade-mesa-drivers-on-ubuntu-linux/ - we have to use the repo kisak/turtle stable for Ubuntu 22.

sudo add-apt-repository ppa:kisak/turtle
sudo apt update
sudo apt upgrade
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libgl1-amber-dri libglapi-mesa
Use 'sudo apt autoremove' to remove them.
Get more security updates through Ubuntu Pro with 'esm-apps' enabled:
  libzvbi-common liburiparser1 libheif1 libmujs1 libavdevice58 ffmpeg
  libpostproc55 libavcodec58 libgstreamer-plugins-bad1.0-0 libavutil56
  libswscale5 freeglut3 libswresample3 libavformat58 libzvbi0 libde265-0
  libavfilter7
Learn more about Ubuntu Pro at https://ubuntu.com/pro
The following NEW packages will be installed:
  mesa-libgallium
The following packages will be upgraded:
  libdrm-amdgpu1 libdrm-common libdrm-intel1 libdrm-nouveau2 libdrm-radeon1
  libdrm2 libegl-mesa0 libgbm1 libgl1-mesa-dri libglx-mesa0 libllvm15
  libvdpau1 libxatracker2 mesa-va-drivers mesa-vdpau-drivers
  mesa-vulkan-drivers vdpau-driver-all
17 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.

income tax filing has become simplified

Income tax filing for us in India has now become further simplified. Bank interest and salary income are automatically being pre-filled via the AIS, so only capital gains if any, from sale of mutual fund units, needs to be entered by us. No need to keep track of any other deductions, since we've now moved to the new tax regime. Also, no need to report gift from parents / sister. So, my workflow was:

1. Download AIS from the income tax site
2. Check the salary component in AIS against the Form16 given to us from the office
3. Verify that the AIS contains the interest statements from all three banks in which I have accounts
4. Download Capital Gains statement from MFCentral 
5. Go through ITR2, entering only minor info (like "secondary address is same as primary address") and the capital gains schedules (which have also become simplified for me since most of the current redemptions are of assets purchased after 2018 and can be entered just as single consolidated figures).

Saturday, June 06, 2026

in-place upgrade Ubuntu 22.04 web server to Ubuntu 24.04

One of our web servers was running Ubuntu 22.04 and showed that an upgrade was available via 'do-release-upgrade'

I took the plunge, running the upgrade via screen in case our connection broke. (The upgrade process also auto-starts sshd on port 1022 also, in case the upgrade needs recovery. But thankfully, I didn't need it.)

Chose the default options every time for retaining the configuration files. Noticed that apache showed a "syntax error" in config files.

After the restart, checked apache status with

sudo systemctl status apache2
× apache2.service - The Apache HTTP Server
     Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Sat 2026-06-06 04:18:46 UTC; 1min 46s ago
       Docs: https://httpd.apache.org/docs/2.4/
    Process: 797 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
        CPU: 28ms
Jun 06 04:18:45 sssihms-web-vm2023 systemd[1]: Starting apache2.service - The Apache HTTP Server...
Jun 06 04:18:45 sssihms-web-vm2023 apachectl[821]: apache2: Syntax error on line 146 of /etc/apache2/apache2.conf: Syntax error on line 3 of /etc/apache2/mods-enabled/php8.1.load:>
Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: apache2.service: Failed with result 'exit-code'.
Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: Failed to start apache2.service - The Apache HTTP Server.

Claude pointed out that Ubuntu 24.04 has php8.3, so loading php8.1 would fail. So, 

sudo a2dismod php8.1

# Install php8.3
sudo apt install php8.3 libapache2-mod-php8.3
# this had already been installed during the upgrade

# Enable the new module
sudo a2enmod php8.3
sudo systemctl restart apache2

All good. Wordpress is also running fine.

Google apps script - Rhino runtime no longer supported

One of our google apps scripts started showing "Execution failed. The Rhino runtime is deprecated and no longer supported."

We needed to migrate the google apps script to V8, based on this - 

Once that was done and the script was re-deployed with v8 runtime changes, the script was working again. 

Monday, June 01, 2026

lobby Raspberry Pi not booting

We had multi-day issues booting the Raspberry Pi in the lobby, which would play a video loop. Disconnecting the USB power and re-connecting it, or taking out the micro-sdcard, checking it for errors on Linux Mint and then re-inserting it would make it work.

Suspecting power supply issues, the current idea is to
1. add a separate switch to turn on power to the Pi after the TVs are powered on.
2. change the power cord as well as the USB power adapter

With these two changes, the Pi booted up without trouble today, we would probably need to monitor it for a week or so to be sure that this has solved the problem.

Friday, May 22, 2026

visuals on planetarium dome getting washed out

Copy-pasting from an email thread - 

Just a follow-up on the issues with washed-out visuals we were facing.

When slowly fading in particular scenes, we can clearly see a point where parts of the scene get washed out, with the projector suddenly jumping in brightness. Looks like this projector, Optoma ZK-507-W, has some poorly executed "high-brightness" handling, which we don't seem to be able to disable. As long as we keep large areas below 50% brightness, small bright areas like stars, trails etc can be viewed with good contrast. 

I think it is some sort of local dimming similar to what is used in some TVs, https://www.academia.edu/88311604/Backlight_local_dimming_algorithm_for_high_contrast_LCD_TV but with very poor results.

Perhaps a warning line can be added at https://paulbourke.net/dome/faq/#projectors for this projector, indicating its poor performance.

To which Paul replied,

I  assume you have tried

1. Turning off the DynamicBlack settings in Display ->  Image setting -> Brightness mode menu.

2. Turning off the HDR/HLG in Display -> Image settings -> Dynamic range menu.

Yes for the 2nd, there was no such menu for the first point. Since my source is not HDR, I also tried "Auto" in addition to "Off".

Also, have tried all the different modes, including "Film" - even that has the noticeable brightness jump once a large enough area has a pixel value high enough (perhaps >50%). 

(Interestingly, some planetarium shows have more problems than others.

Eg. Unseen Earth, https://www.unseenearth.eu/ - has some drone shots etc which look terrible on our dome if I don't mask out most of it to <50%

Mars - the Ultimate Voyage - https://www.bellmuseum.umn.edu/mars-ultimate-voyage/ - has gamma settings, lighting etc such that most of the show looks pretty good on our dome.

In our own productions using OpenSpace, https://www.openspaceproject.com/ , there seems to be a difference between scenes rendered on Mac and scenes rendered on PC (Linux). Maybe gamma differences.

These micro-organisms are rendered fine, while I had to darken these scenes rendered on Mac for Jupiter, Saturn and Europa to not become completely washed out.

Interestingly, this scene, rendered on Linux, is fine.)

Wednesday, May 20, 2026

OpenSpace builds on Github actions getting "terminated" - exit code 143

Some of my build github action workflows were consistently getting terminated after 10 minutes of run time or so. 

gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** wait: No child processes.  Stop.
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3607: ext/spice/CMakeFiles/spice.dir/src/common/dafopw_c.c.o] Terminated
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3635: ext/spice/CMakeFiles/spice.dir/src/common/dafps_c.c.o] Terminated
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3663: ext/spice/CMakeFiles/spice.dir/src/common/dafrcr.c.o] Terminated
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3649: ext/spice/CMakeFiles/spice.dir/src/common/dafra.c.o] Terminated
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3691: ext/spice/CMakeFiles/spice.dir/src/common/dafrda_c.c.o] Terminated
gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3565: ext/spice/CMakeFiles/spice.dir/src/common/dafgsr_c.c.o] Terminated
gmake[2]: *** [modules/globebrowsing/ext/geos/CMakeFiles/geos.dir/build.make:2514: modules/globebrowsing/ext/geos/CMakeFiles/geos.dir/src/io/StringTokenizer.cpp.o] Terminated
Error: Process completed with exit code 143.

Gemini - via aistudio.google.com free plan - pointed out that 

Exit code 143 indicates that a process was terminated by a SIGTERM signal. In GitHub Actions, this almost always happens when the build runner runs out of memory (OOM) and the operating system or the GitHub runner agent terminates the compiler processes to save the system.
Compiling large C++ projects like OpenSpace with heavy submodules (such as Assimp/Draco, Boost, and Spicelib) using GCC 13 can easily exceed the 7 GB RAM limit of the standard GitHub-hosted runner, especially when building in parallel.
Here are the two most effective ways to resolve this:
1. Limit Build Parallelism
By default, cmake --build . --parallel attempts to use all available virtual CPUs. While GitHub runners have 2 vCPUs, compiling templates in parallel can still exceed 7 GB of RAM.
Limit the concurrent build jobs to 2 etc etc

So I changed 
 cmake --build . --parallel
to
 cmake --build . --parallel 3

like the workflow which was working, and that solved the problem. Told Gemini that, and the reply was,

You are correct. Standard GitHub-hosted runners for public repositories were updated to 4 vCPUs and 16 GB of RAM.
When you run cmake --build . --parallel without specifying a limit, it attempts to spawn 4 compilation jobs. With heavy C++ headers and templates, 4 concurrent GCC processes can still spike past the 16 GB memory limit, triggering the termination.
Limiting the build to --parallel 3 (or --parallel 2 if memory usage remains high) is a solid approach to keep the peak memory consumption within the runner's limits.

Then I asked Gemini,
Same codebase, same runner.
cmake --build . --parallel causes termination.
make -j $(nproc) does not cause termination, why?

Then Gemini changed the answer to specify that parallel without a number means unlimited parallel jobs - 

The difference lies in how GNU Make and CMake interpret the omission of a job
number.

1. cmake --build . --parallel maps to unlimited jobs

When you run cmake --build . --parallel without specifying a number, CMake
translates this to the underlying build tool's default parallel flag.

For GNU Make, this translates to running make -j (with no number). In GNU Make,
omitting the number instructs it to run with infinite/unlimited parallel jobs.

Instead of spawning 4 compiler processes (the number of CPU cores), it will try
to compile dozens of source files simultaneously. This immediately exhausts
the 16 GB of RAM and triggers the OOM killer.

2. make -j $(nproc) strictly limits the jobs

When you run make -j $(nproc):

1.  $(nproc) evaluates to 4.
2.  The command becomes make -j 4.
3.  GNU Make is strictly limited to running a maximum of 4 compiler processes at
    any given time, which usually fits within the 16 GB RAM limit.

Solution

If you want to use the CMake CLI wrapper safely, you should explicitly pass the
CPU count to it, similar to what you did with make:

cmake --build . --parallel $(nproc)




Saturday, May 16, 2026

Moodle course order - manual sorting option vanished

One of our Moodle instances had the manual sorting option for courses disappear, at server/course/management.php

ChatGPT gave some hallucinatory suggestions, Gemini via aistudio.google.com gave the first suggestion of removing pagination by using the dropdown to show all courses instead of only the first 20 - the number of courses had become 21. Even after doing this, the sorting option did not return. But once I added a dummy category and some dummy courses and then deleted them, the sorting option returned. Gemini said, "When you created the dummy category and dummy courses, and then deleted them,
you triggered a core Moodle background script called fix_course_sortorder()." So that has fixed it.

Reading up on this function, it looks like for very early versions, this function was also buggy - https://moodle.org/mod/forum/discuss.php?d=126896 - but in current versions, this seems to be rewritten, like in the current main branch.

Tuesday, May 12, 2026

msmtp instead of mail

Noticed only today that for some scripts on our internal server, the mail command was commented out and replaced by msmtp like 

#mail -s "Automatic Encoded file Statistics for : date" -t $MAIL_COPY $MAIL_DESTINATION < $MAIL_FILE 

{ echo "To: $MAIL_DESTINATION"; echo "Cc: $MAIL_COPY"; echo "Subject: Automatic Encoded file Statistics: $(date)"; echo; cat "$MAIL_FILE"; } | /usr/bin/msmtp -t

ChatGPT explains,

Yes, on many modern Linux systems, using msmtp directly is now preferred over the traditional mail command, especially for scripts and automated systems.

The reason is that the mail command historically depended on a local Mail Transfer Agent (MTA) such as:

Sendmail
Postfix
Exim

Many newer minimal/server/container installations no longer install or configure a full local MTA.

This was mentioned in this previous post, which mentions a replacement of ssmtp with msmtp. 

Monday, May 11, 2026

UTM and the mouse

When switching between machines using a KVM switch, found that Ubuntu 24.04 running on UTM on Mac, full screen, would ask, "Allow UTM to use the mouse" or something like that. We should choose cancel instead of Yes. If we choose Yes, the mouse cursor would not be seen, though mouse-clicks would be registered correctly on Ubuntu 24.04. If we choose cancel, the mouse would work properly.

Saturday, April 25, 2026

two or three ways to make LLMs help with Blender scene creation

If we directly ask Gemini to create a Blender file, it says it can't, but it can generate a python script which can be run inside Blender to create a scene. This is useful if we need to generate repetitive visuals or something which can be programmatically generated. Example, CirclesVideos.blend - the prompt for Gemini 3, via the free tier of aistudio.google.com, is below.

Can you now generate a python script to create a blender scene with

  1. Fisheye 180 degree equidistant camera in the centre, pointing straight up

  2. equidistant circular screens subtending just less than 40 degrees so that they do not touch, having a small gap between them, 9 circular screens nearest the equator, and more rows of similarly sized circular screens, but fewer as we go up the hemisphere since the circumference of higher latitudes would be lower

  3. The lowest row of circular screens must be completely above the horizon for the viewport of the 180 degree fisheye equidistant camera

  4. The screens should fade in one by one starting from the lowest row, left to right, sequentially, one after the other, with a one second fade in time. So, circle one fades in by the 00:00:01 mark, then circle two fades in and finishes fading in by the 00:00:02 mark, and so on. The total duration of the animation can be one second greater than the time required to fade in all the circles.

  5. Each circular screen should have an image texture mapped to a unique file - named seq01.jpg for circle one, seq02.jpg for circle two and so on, with the jpg textures located in a subdirectory of the project directory called seq. The image texture would be square in shape, but should be zoomed in so that the entire circle is filled with the texture.


This resulted in the following:

Here is the complete Python script to generate your exact fulldome scene.

It handles the math for perfectly spacing the rows along the hemisphere without
overlapping, scales the circular cutouts to exactly 38 degrees (leaving
a 2-degree gap so they don't touch), sets up the sequential 1-second fade-ins,
and auto-links your seq/seqXX.jpg files.

How to use it:

1.  Open a new Blender file.
2.  Go to the Scripting tab.
3.  Click New, paste the code below, and click the Run Script (Play) button.
4.  Set your 3D Viewport to Rendered View to see the fade-in math and image
    textures working perfectly.

import bpy
import math

# ==========================================
# 1. CLEANUP PREVIOUS SCENE
# ==========================================
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete(use_global=False)

# ==========================================
# 2. SCENE & ANIMATION SETTINGS
# ==========================================
FPS = 24
TOTAL_SCREENS = 14
FADE_DURATION = FPS * 1 # 1 second fade in per screen

scene = bpy.context.scene
scene.render.engine = 'CYCLES'
scene.render.resolution_x = 2048
scene.render.resolution_y = 2048
scene.render.fps = FPS

# Total duration = 1 second per screen, plus 1 second at the end
TOTAL_FRAMES = (TOTAL_SCREENS * FADE_DURATION) + FADE_DURATION
scene.frame_start = 1
scene.frame_end = TOTAL_FRAMES

# ==========================================
# 3. FULLDOME CAMERA (Equidistant)
# ==========================================
cam_data = bpy.data.cameras.new("Fulldome_Camera")
cam_data.type = 'PANO'
cam_data.panorama_type = 'FISHEYE_EQUIDISTANT'
cam_data.fisheye_fov = math.radians(180)

cam_obj = bpy.data.objects.new("Fulldome_Camera", cam_data)
bpy.context.collection.objects.link(cam_obj)
scene.camera = cam_obj

# Camera at center (0,0,0) pointing straight UP (Zenith)
cam_obj.location = (0, 0, 0)
cam_obj.rotation_euler = (math.radians(180), 0, 0)

# Dark Background
world = scene.world
world.use_nodes = True
bg_node = world.node_tree.nodes.get("Background")
if bg_node:
    bg_node.inputs[0].default_value = (0.01, 0.01, 0.02, 1)

# ==========================================
# 4. MATH: SCREEN PLACEMENTS
# ==========================================
# Dome Radius
R = 10.0

# To subtend exactly 38 degrees (leaving a gap under 40deg),
# the physical radius of the plane at distance R is R * tan(19 degrees)
screen_radius = R * math.tan(math.radians(19))

# Calculate positions (Elevation and Azimuth)
# Row 1: 9 screens at 20 degrees elevation. 
# (Bottom edge is 20 - 19 = 1 degree above horizon. Perfect!)
# Row 2: 5 screens at 60 degrees elevation.
# (Bottom edge is 60 - 19 = 41 degrees. Top of Row 1 is 39. Gap of 2 degrees!)

screen_data = []

# Row 1 (9 Screens, lowest row)
for i in range(9):
    azimuth = i * (360.0 / 9.0)
    screen_data.append({'el': 20.0, 'az': azimuth})

# Row 2 (5 Screens, upper row)
# Offset azimuth slightly so they interlock beautifully
for i in range(5):
    azimuth = i * (360.0 / 5.0) + 36.0 
    screen_data.append({'el': 60.0, 'az': azimuth})

# ==========================================
# 5. MATERIAL GENERATOR
# ==========================================
def create_fade_circle_material(index, start_frame, end_frame):
    mat_name = f"Mat_Screen_{index:02d}"
    mat = bpy.data.materials.new(name=mat_name)
    mat.use_nodes = True
    mat.blend_method = 'CLIP' # Helps viewport transparency
    nodes = mat.node_tree.nodes
    links = mat.node_tree.links
    nodes.clear()

    # Base Nodes
    output = nodes.new(type='ShaderNodeOutputMaterial')
    output.location = (1000, 0)

    # Fade Setup (Mix Shader 2)
    fade_mix = nodes.new(type='ShaderNodeMixShader')
    fade_mix.location = (800, 0)
    
    fade_trans = nodes.new(type='ShaderNodeBsdfTransparent')
    fade_trans.location = (600, 100)

    # Cutout Setup (Mix Shader 1)
    cutout_mix = nodes.new(type='ShaderNodeMixShader')
    cutout_mix.location = (600, -100)
    
    cutout_trans = nodes.new(type='ShaderNodeBsdfTransparent')
    cutout_trans.location = (400, 0)

    # Video/Image Setup
    emission = nodes.new(type='ShaderNodeEmission')
    emission.location = (400, -200)
    emission.inputs['Strength'].default_value = 1.5

    img_tex = nodes.new(type='ShaderNodeTexImage')
    img_tex.location = (100, -200)
    # Assuming relative path in a "seq" folder next to your .blend file
    img_path = f"//seq/seq{index:02d}.jpg"
    
    # Try to load the image if it exists, otherwise leave placeholder
    try:
        img = bpy.data.images.load(img_path)
        img_tex.image = img
    except:
        print(f"Warning: Could not load {img_path}. Placeholder created.")

    # Math for the Circle Mask
    tex_coord = nodes.new(type='ShaderNodeTexCoord')
    tex_coord.location = (-300, 0)

    distance = nodes.new(type='ShaderNodeVectorMath')
    distance.operation = 'DISTANCE'
    distance.location = (-100, 0)
    distance.inputs[1].default_value = (0.5, 0.5, 0.0)

    less_than = nodes.new(type='ShaderNodeMath')
    less_than.operation = 'LESS_THAN'
    less_than.location = (100, 0)
    less_than.inputs[1].default_value = 0.5

    # Wire it all up
    links.new(tex_coord.outputs['UV'], distance.inputs[0])
    links.new(distance.outputs['Value'], less_than.inputs[0])
    
    links.new(tex_coord.outputs['UV'], img_tex.inputs['Vector'])
    links.new(img_tex.outputs['Color'], emission.inputs['Color'])

    links.new(less_than.outputs['Value'], cutout_mix.inputs['Fac'])
    links.new(cutout_trans.outputs['BSDF'], cutout_mix.inputs[1])
    links.new(emission.outputs['Emission'], cutout_mix.inputs[2])

    links.new(fade_trans.outputs['BSDF'], fade_mix.inputs[1])
    links.new(cutout_mix.outputs['Shader'], fade_mix.inputs[2])
    
    links.new(fade_mix.outputs['Shader'], output.inputs['Surface'])

    # --- ANIMATE THE FADE ---
    # At start of animation: completely transparent
    fade_mix.inputs['Fac'].default_value = 0.0
    fade_mix.inputs['Fac'].keyframe_insert(data_path="default_value", frame=1)
    
    # Just before its turn: still transparent
    fade_mix.inputs['Fac'].default_value = 0.0
    fade_mix.inputs['Fac'].keyframe_insert(data_path="default_value", frame=start_frame)

    # 1 second later: fully visible
    fade_mix.inputs['Fac'].default_value = 1.0
    fade_mix.inputs['Fac'].keyframe_insert(data_path="default_value", frame=end_frame)

    return mat

# ==========================================
# 6. SPAWN SCREENS
# ==========================================
for i, data in enumerate(screen_data):
    el_rad = math.radians(data['el'])
    az_rad = math.radians(data['az'])
    
    # Convert polar to Cartesian XYZ
    z = R * math.sin(el_rad)
    r_xy = R * math.cos(el_rad)
    x = r_xy * math.cos(az_rad)
    y = r_xy * math.sin(az_rad)
    
    # 1. Create the Plane
    bpy.ops.mesh.primitive_plane_add(size=1.0, location=(x, y, z))
    plane = bpy.context.active_object
    
    # Index is i+1 (so it starts at 1)
    idx = i + 1
    plane.name = f"Screen_{idx:02d}"
    
    # 2. Scale to exact angular size (38 degrees)
    # We multiply by 2 because screen_radius is from the center to edge.
    plane.scale = (screen_radius * 2, screen_radius * 2, 1.0)
    
    # 3. Add Track To Constraint (Always face camera)
    track = plane.constraints.new(type='TRACK_TO')
    track.target = cam_obj
    track.track_axis = 'TRACK_Z'
    track.up_axis = 'UP_Y'
    
    # 4. Create and assign animated material
    start_f = (i * FADE_DURATION) + 1
    end_f = start_f + FADE_DURATION
    
    mat = create_fade_circle_material(idx, start_f, end_f)
    plane.data.materials.append(mat)

print(f"Fulldome scene generated! Total screens: {TOTAL_SCREENS}. Animation frames: {TOTAL_FRAMES}")



LLMs can even give us alternate ideas. For example, this conversation, 

Prompt for Gemini,

I'd like to show a montage of videos in Blender to be rendered for fulldome (180
degree fisheye equisolid camera). The script is "We constantly seek happiness in
the form of pleasure, peace, or the satisfaction of curiosity." My current idea
is to depict a pyramid with video textures on the four sides depicting
"pleasure, peace, or the satisfaction of curiosity." Any other ideas?

Which resulted in this reply - 

Designing for a fulldome (180° fisheye) is a completely different beast than
designing for a flat screen.

Your pyramid idea is conceptually strong, but in a fulldome environment, a
single object floating in the center of the screen can sometimes feel a bit
"small," leaving the vast periphery of the dome empty. In fulldome, you want to
surround the viewer or give them a sense of scale and immersion.

Here are a few alternative concepts specifically tailored to make the most of
the 180° fisheye camera in Blender, incorporating your video montages and
script.

1. The Canopy of Spheres (Focus on "Looking Up")

  - The Visual: The camera sits at the base of a massive, glowing, abstract tree
    or DNA-like structure that stretches up to the "zenith" (the dead center of
    your fisheye circle, which is the top of the dome). Instead of leaves,
    giant, slow-moving translucent bubbles or glass orbs hang above the viewer.
  - The Montage: Inside these orbs are your videos.
      - Pleasure videos can be in warm, rapidly pulsing orange/red orbs.
      - Peace videos in slow-spinning, tranquil blue/teal orbs.
      - Curiosity videos in geometric, bright white/purple orbs.
  - Why it works for Fulldome: Audiences in planetariums naturally look up. By
    placing the anchor of the structure at the bottom edge of the fisheye (the
    springline) and having it branch out overhead, you create a massive sense of
    scale and immersion.

2. The Monolith Portals (Surrounding the Viewer)

  - The Visual: Instead of looking at a pyramid from the outside, place the
    camera inside a massive, open space (like a serene desert, a starry void, or
    a calm ocean). Standing out of the ground, surrounding the camera in
    a 360-degree radius (or semi-circle), are three monolithic, towering
    screens/portals.
  - The Montage: As the narrator speaks, the camera slowly rotates.
      - "Pleasure" lights up the monolith to the left, reflecting its video off
        the glossy floor.
      - "Peace" lights up the center monolith.
      - "Curiosity" lights up the right monolith.
  - Why it works for Fulldome: It places the viewer inside the geometry rather
    than watching it from afar. Wide, towering objects that wrap around the
    peripheral vision (the edges of the fisheye circle) look incredibly imposing
    and immersive in a dome.

3. The Tunnel of Seeking (Forward Motion)

  - The Visual: "Constantly seek" implies forward momentum. The camera is moving
    slowly and continuously forward through a massive, abstract, hexagonal or
    circular tunnel.
  - The Montage: The walls of the tunnel are made of floating video panels.
      - As the word "Pleasure" is spoken, panels whiz by showing vibrant,
        sensory clips.
      - For "Peace," the tunnel widens, the movement slows slightly, and the
        panels show calming nature or meditative clips.
      - For "Curiosity," the tunnel breaks apart into a constellation of
        floating screens showing science, exploration, or complex data.
  - Why it works for Fulldome: Forward motion (called vection) is the single
    most powerful optical illusion in a fulldome. A slow, steady forward push
    through a tunnel makes the audience feel like they are physically moving
    through space.

4. The Fluid Kaleidoscope (Abstract Blending)

  - The Visual: Instead of harsh geometric shapes (like a pyramid), map your
    videos onto fluid, undulating mesh surfaces—like ribbons of silk or waves of
    water—that drift right over the camera lens.
  - The Montage:
      - A vibrant ribbon of video (Pleasure) swoops across the dome.
      - It is intercepted by a slow-moving, massive, calm wave of video (Peace)
        that washes over the whole screen.
      - Finally, a sharply focused, fractal-like geometric burst of video
        screens (Curiosity) shatters through the peaceful wave.
  - Why it works for Fulldome: Sharp edges and straight lines (like a pyramid)
    are heavily curved and distorted by fisheye lenses. Organic, fluid shapes
    hide this distortion and feel incredibly natural and dreamlike in a dome.

5. If You Stick With the Pyramid: The Interior Pyramid

If you really love the pyramid motif, flip the perspective. Place the camera
inside a colossal pyramid.

  - The Visual: The camera looks up toward the apex of the pyramid (which sits
    in the center of your fisheye render). The walls of the pyramid surround the
    viewer.
  - The Montage: The three visible interior walls of the pyramid become massive
    projection mapping surfaces. One wall illuminates with "Pleasure," the next
    with "Peace," and the third with "Curiosity." As the script concludes, the
    apex of the pyramid opens up, revealing a starry sky or a bright light
    (representing the overarching "Happiness").

💡 Quick Blender Fulldome Tips:

1.  Camera Setup: Set your camera to Panoramic -> Fisheye Equisolid. Field of
    View: 180 degrees. Render resolution must be perfectly square (e.g., 4096
    x 4096).
2.  The Sweet Spot: In a dome, the audience is usually looking about 30 to 45
    degrees up from the bottom edge. Don't put your most important video exactly
    in the dead center (zenith), as it forces the audience to crane their necks
    straight up.
3.  Avoid Fast Camera Pans: Panning left or right in a dome causes severe motion
    sickness. Keep camera movements to slow forward/backward tracking, or very
    slow rotations. Let the objects (the video panels) move around the camera
    instead.

Tuesday, April 21, 2026

false alarm emails from github pages

We got three emails from github pages in March and April, saying Please verify your Pages domain (ACTION NEEDED) - "We are unable to find a verification TXT record  .... Please verify your domain within 7 days by following our instructions before it becomes unverified."

Found in each case by going to the github settings page for gh-pages, that the domain was verified OK, and the verification txt record was also fine. 

Since this has only happened for a domain which has DNS operated by Microsoft - Azure DNS - I guess the errors were DNS failures from Azure.

Friday, April 17, 2026

Blender 5 VSE difference for extending clips

 Blender 5 Video Sequence Editor (VSE) handles extension of clips differently than earlier versions. In earlier versions, if we extended a clip past its end by dragging its boundary, it would create still frames from the last frame of the clip, and if we used a speed control modifier, we could stretch clips both to extend them as well as to make them shorter. With Blender 5, we can't do that directly. Now, we need to go to clip properties, increase duration of the clip there. Just using speed control is no longer enough. Via  https://blender.stackexchange.com/questions/69679/extend-video-strip-in-vse

Sunday, April 05, 2026

can't have a show all apps button on old Realme phone

There was no way to install a button to "show all apps" like on my Samsung M34, on the old Realme phone: in settings -> home screen, there is an option to show drawer instead of home screen icons - all the apps are listed in the drawer. Chose that instead.

Saturday, April 04, 2026