in-place upgrade Ubuntu 22.04 web server to Ubuntu 24.04

One of our web servers was running Ubuntu 22.04 and showed that an upgrade was available via 'do-release-upgrade'

I took the plunge, running the upgrade via screen in case our connection broke. (The upgrade process also auto-starts sshd on port 1022 also, in case the upgrade needs recovery. But thankfully, I didn't need it.)

Chose the default options every time for retaining the configuration files. Noticed that apache showed a "syntax error" in config files.

After the restart, checked apache status with

sudo systemctl status apache2

× apache2.service - The Apache HTTP Server

     Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; preset: enabled)

     Active: failed (Result: exit-code) since Sat 2026-06-06 04:18:46 UTC; 1min 46s ago

       Docs: https://httpd.apache.org/docs/2.4/

    Process: 797 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)

        CPU: 28ms

Jun 06 04:18:45 sssihms-web-vm2023 systemd[1]: Starting apache2.service - The Apache HTTP Server...

Jun 06 04:18:45 sssihms-web-vm2023 apachectl[821]: apache2: Syntax error on line 146 of /etc/apache2/apache2.conf: Syntax error on line 3 of /etc/apache2/mods-enabled/php8.1.load:>

Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE

Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: apache2.service: Failed with result 'exit-code'.

Jun 06 04:18:46 sssihms-web-vm2023 systemd[1]: Failed to start apache2.service - The Apache HTTP Server.

Claude pointed out that Ubuntu 24.04 has php8.3, so loading php8.1 would fail. So, 

sudo a2dismod php8.1

# Install php8.3

sudo apt install php8.3 libapache2-mod-php8.3

# this had already been installed during the upgrade

# Enable the new module

sudo a2enmod php8.3

sudo systemctl restart apache2

All good. Wordpress is also running fine.

lobby Raspberry Pi not booting

We had multi-day issues booting the Raspberry Pi in the lobby, which would play a video loop. Disconnecting the USB power and re-connecting it, or taking out the micro-sdcard, checking it for errors on Linux Mint and then re-inserting it would make it work.

Suspecting power supply issues, the current idea is to

1. add a separate switch to turn on power to the Pi after the TVs are powered on.

2. change the power cord as well as the USB power adapter

With these two changes, the Pi booted up without trouble today, we would probably need to monitor it for a week or so to be sure that this has solved the problem.

visuals on planetarium dome getting washed out

Copy-pasting from an email thread - 

Just a follow-up on the issues with washed-out visuals we were facing.

When slowly fading in particular scenes, we can clearly see a point where parts of the scene get washed out, with the projector suddenly jumping in brightness. Looks like this projector, Optoma ZK-507-W, has some poorly executed "high-brightness" handling, which we don't seem to be able to disable. As long as we keep large areas below 50% brightness, small bright areas like stars, trails etc can be viewed with good contrast. 

I think it is some sort of local dimming similar to what is used in some TVs, https://www.academia.edu/88311604/Backlight_local_dimming_algorithm_for_high_contrast_LCD_TV but with very poor results.

Perhaps a warning line can be added at https://paulbourke.net/dome/faq/#projectors for this projector, indicating its poor performance.

To which Paul replied,

I  assume you have tried

1. Turning off the DynamicBlack settings in Display ->  Image setting -> Brightness mode menu.

2. Turning off the HDR/HLG in Display -> Image settings -> Dynamic range menu.

Yes for the 2nd, there was no such menu for the first point. Since my source is not HDR, I also tried "Auto" in addition to "Off".

Also, have tried all the different modes, including "Film" - even that has the noticeable brightness jump once a large enough area has a pixel value high enough (perhaps >50%). 

(Interestingly, some planetarium shows have more problems than others.

Eg. Unseen Earth, https://www.unseenearth.eu/ - has some drone shots etc which look terrible on our dome if I don't mask out most of it to <50%

Mars - the Ultimate Voyage - https://www.bellmuseum.umn.edu/mars-ultimate-voyage/ - has gamma settings, lighting etc such that most of the show looks pretty good on our dome.

In our own productions using OpenSpace, https://www.openspaceproject.com/ , there seems to be a difference between scenes rendered on Mac and scenes rendered on PC (Linux). Maybe gamma differences.

These micro-organisms are rendered fine, while I had to darken these scenes rendered on Mac for Jupiter, Saturn and Europa to not become completely washed out.

Interestingly, this scene, rendered on Linux, is fine.)

OpenSpace builds on Github actions getting "terminated" - exit code 143

Some of my build github action workflows were consistently getting terminated after 10 minutes of run time or so. 

gmake[2]: *** Waiting for unfinished jobs....

gmake[2]: *** wait: No child processes.  Stop.

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3607: ext/spice/CMakeFiles/spice.dir/src/common/dafopw_c.c.o] Terminated

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3635: ext/spice/CMakeFiles/spice.dir/src/common/dafps_c.c.o] Terminated

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3663: ext/spice/CMakeFiles/spice.dir/src/common/dafrcr.c.o] Terminated

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3649: ext/spice/CMakeFiles/spice.dir/src/common/dafra.c.o] Terminated

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3691: ext/spice/CMakeFiles/spice.dir/src/common/dafrda_c.c.o] Terminated

gmake[2]: *** [ext/spice/CMakeFiles/spice.dir/build.make:3565: ext/spice/CMakeFiles/spice.dir/src/common/dafgsr_c.c.o] Terminated

gmake[2]: *** [modules/globebrowsing/ext/geos/CMakeFiles/geos.dir/build.make:2514: modules/globebrowsing/ext/geos/CMakeFiles/geos.dir/src/io/StringTokenizer.cpp.o] Terminated

Error: Process completed with exit code 143.

Gemini - via aistudio.google.com free plan - pointed out that 

Exit code 143 indicates that a process was terminated by a SIGTERM signal. In GitHub Actions, this almost always happens when the build runner runs out of memory (OOM) and the operating system or the GitHub runner agent terminates the compiler processes to save the system.

Compiling large C++ projects like OpenSpace with heavy submodules (such as Assimp/Draco, Boost, and Spicelib) using GCC 13 can easily exceed the 7 GB RAM limit of the standard GitHub-hosted runner, especially when building in parallel.

Here are the two most effective ways to resolve this:

1. Limit Build Parallelism

By default, cmake --build . --parallel attempts to use all available virtual CPUs. While GitHub runners have 2 vCPUs, compiling templates in parallel can still exceed 7 GB of RAM.

Limit the concurrent build jobs to 2 etc etc

So I changed 
 cmake --build . --parallel

to
 cmake --build . --parallel 3

like the workflow which was working, and that solved the problem. Told Gemini that, and the reply was,

You are correct. Standard GitHub-hosted runners for public repositories were updated to 4 vCPUs and 16 GB of RAM.

When you run cmake --build . --parallel without specifying a limit, it attempts to spawn 4 compilation jobs. With heavy C++ headers and templates, 4 concurrent GCC processes can still spike past the 16 GB memory limit, triggering the termination.

Limiting the build to --parallel 3 (or --parallel 2 if memory usage remains high) is a solid approach to keep the peak memory consumption within the runner's limits.

Then I asked Gemini,

Same codebase, same runner.
cmake --build . --parallel causes termination.
make -j $(nproc) does not cause termination, why?

Then Gemini changed the answer to specify that parallel without a number means unlimited parallel jobs - 

The difference lies in how GNU Make and CMake interpret the omission of a job

number.

1. cmake --build . --parallel maps to unlimited jobs

When you run cmake --build . --parallel without specifying a number, CMake

translates this to the underlying build tool's default parallel flag.

For GNU Make, this translates to running make -j (with no number). In GNU Make,

omitting the number instructs it to run with infinite/unlimited parallel jobs.

Instead of spawning 4 compiler processes (the number of CPU cores), it will try

to compile dozens of source files simultaneously. This immediately exhausts

the 16 GB of RAM and triggers the OOM killer.

2. make -j $(nproc) strictly limits the jobs

When you run make -j $(nproc):

1.  $(nproc) evaluates to 4.

2.  The command becomes make -j 4.

3.  GNU Make is strictly limited to running a maximum of 4 compiler processes at

    any given time, which usually fits within the 16 GB RAM limit.

Solution

If you want to use the CMake CLI wrapper safely, you should explicitly pass the

CPU count to it, similar to what you did with make:

cmake --build . --parallel $(nproc)

Moodle course order - manual sorting option vanished

One of our Moodle instances had the manual sorting option for courses disappear, at server/course/management.php

ChatGPT gave some hallucinatory suggestions, Gemini via aistudio.google.com gave the first suggestion of removing pagination by using the dropdown to show all courses instead of only the first 20 - the number of courses had become 21. Even after doing this, the sorting option did not return. But once I added a dummy category and some dummy courses and then deleted them, the sorting option returned. Gemini said, "When you created the dummy category and dummy courses, and then deleted them,

you triggered a core Moodle background script called fix_course_sortorder()." So that has fixed it.

Reading up on this function, it looks like for very early versions, this function was also buggy - https://moodle.org/mod/forum/discuss.php?d=126896 - but in current versions, this seems to be rewritten, like in the current main branch.

msmtp instead of mail

Noticed only today that for some scripts on our internal server, the mail command was commented out and replaced by msmtp like 

#mail -s "Automatic Encoded file Statistics for : date" -t $MAIL_COPY $MAIL_DESTINATION < $MAIL_FILE 

{ echo "To: $MAIL_DESTINATION"; echo "Cc: $MAIL_COPY"; echo "Subject: Automatic Encoded file Statistics: $(date)"; echo; cat "$MAIL_FILE"; } | /usr/bin/msmtp -t

ChatGPT explains,

Yes, on many modern Linux systems, using msmtp directly is now preferred over the traditional mail command, especially for scripts and automated systems.

The reason is that the mail command historically depended on a local Mail Transfer Agent (MTA) such as:

Sendmail

Postfix

Exim

Many newer minimal/server/container installations no longer install or configure a full local MTA.

This was mentioned in this previous post, which mentions a replacement of ssmtp with msmtp. 

UTM and the mouse

When switching between machines using a KVM switch, found that Ubuntu 24.04 running on UTM on Mac, full screen, would ask, "Allow UTM to use the mouse" or something like that. We should choose cancel instead of Yes. If we choose Yes, the mouse cursor would not be seen, though mouse-clicks would be registered correctly on Ubuntu 24.04. If we choose cancel, the mouse would work properly.