Upgrading a dot net + Wordpress server from Ubuntu 22.04 to 24.04

Instead of a fresh install on a fresh server, I tried the alternate method of upgrading in-place for our AWS server running a dot net app and a couple of Wordpress instances. A snapshot was taken by PB just before the upgrade, to be able to roll back in case of a failed upgrade. The snapshot took around half an hour to complete. 

Running do-release-upgrade resulted in the error "Please install all available updates for your release before upgrading"

https://askubuntu.com/questions/1231570/cant-upgrade-ubuntu-18-04-to-20-04-because-of-please-install-all-available-upd

Apparently, just apt upgrade is not enough, we have to do a dist-upgrade to upgrade the packages which have been "held back". https://itsfoss.com/apt-get-upgrade-vs-dist-upgrade/

sudo apt update

sudo apt upgrade

sudo apt dist-upgrade

sudo apt autoremove

sudo do-release-upgrade

During the upgrade, it warned that the upgrade is being done over ssh, and that it is starting an ssh daemon on port 1022 also. But I didn't have to use that port or request PB to open that port in the AWS firewall. There were several warnings about configuration files existing, and the default was to not overwrite them - which is what I chose. The upgrade itself completed in ten minutes, and a reboot was uneventful except for an apache error.

service apache2 status

showed that the web server was down, an error in the conf file, php8.1 module not found. Now, I knew that 24.04 had php8.3 and not 8.1, so the error was understandable. I thought I would need to again follow the LAMP stack install - https://www.digitalocean.com/community/tutorials/how-to-install-lamp-stack-on-ubuntu - but no - just had to do

 sudo a2dismod php8.1

 sudo a2enmod php8.3

and 

 sudo systemctl restart apache2

The next issue was that the dot net service was not running. It was stuck in a restart loop. Disabled the service we had created as per this post - https://hnsws.blogspot.com/2023/05/dot-net-core-ubuntu-linux-server.html

Running the command to be executed on the command line  

dotnet --version

Command 'dotnet' not found, but can be installed with:

sudo snap install dotnet-sdk       # version 8.0.403, or

sudo apt  install dotnet-host-8.0  # version 8.0.10-0ubuntu1~24.04.1

sudo apt  install dotnet-host      # version 6.0.135-0ubuntu1~22.04.1

sudo apt  install dotnet-host-7.0  # version 7.0.119-0ubuntu1~22.04.1

https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu-install?tabs=dotnet8&pivots=os-linux-ubuntu-2404

Also, we had earlier pinned dot net to install from Microsoft's servers, but with Ubuntu 24, we're supposed to use Ubuntu's repositories. So, deleted the lines mentioned in this post,

https://hnsws.blogspot.com/2023/09/dot-net-app-on-linux-broke-forbidden.html

sudo nano /etc/apt/preferences

Package: dotnet-* aspnetcore-* netstandard-*

Pin: origin "packages.microsoft.com"

Pin-Priority: 999

(deleted those lines)

sudo apt-get install -y aspnetcore-runtime-8.0

/usr/bin/dotnet /home/path/to/OurWebApi.dll

This gave the error

You must install or update .NET to run this application.
Framework: 'Microsoft.NETCore.App', version '7.0.0' (x64)

.NET location: /usr/lib/dotnet/

The following frameworks were found:

  8.0.10

and gave this URL for framework 7, https://dotnet.microsoft.com/en-us/download/dotnet/7.0/runtime?cid=getdotnetcore&arch=x64

which said framework 7 had reached end of life. So, requested the maintainers to target dot net 8.0 instead of 7.0, which they did in a few hours, after which I restarted the service, and the app was running again.

Deleting many GBs of course content from Moodle

Deleting large courses had a tendency to make one of our Moodle instances time out. As I found out by trial and error, and as mentioned at https://moodle.org/mod/forum/discuss.php?d=207712

in such cases, we have to delete module by module, I guess. That worked, but found that for one of our instances, there were still many GB of data in the moodledata filedir, so probably it was data which was not referenced in the database, due to earlier site cloning. 

https://moodle.org/mod/forum/discuss.php?d=376475

cd /the/moodle/directory

moosh file-dbcheck > /path/to/filestodelete.txt

Then I just edited the filestodelete.txt to pre-pend sudo rm -f to each filename in the list, and ran it to get rid of around 8 GB. Completed in less than a minute. 

checking ssd size

After purchasing some SSDs at a 50% discount on Amazon, I wanted to check if they are really 256 GB SSDs. 

https://superuser.com/questions/470949/how-do-i-create-a-1gb-random-file-in-linux

With the aim of filling up at least 100 GB of space on the disk, I just tried

truncate --size 1G foo

shred --iterations 1 foo

That completed in less than a minute. But the same thing with 100G instead of 1G - truncate completed instantly, but shred was going to take time. After ten minutes or so, I cancelled it. Then tried copying over the 100 GB file from one disk to another etc, that was also not complaining, so I concluded that the disk was fine. 

Moodle - Activity Completion Restriction not visible

There was a request for support, that on one of our Moodle instances, we do not see the 'Activity Completion' as an option in 'Restriction'.

From the documentation page,

https://docs.moodle.org/405/en/Restrict_access_settings

In Site administration > Plugins > Availability restrictions > Manage restrictions you can enable or disable (Hide/Show) any of the individual restriction types for use throughout the site. This was already enabled for the Moodle instance.

Maybe if you choose the first activity of a course, this option may not be available. Or if the activity is of type forum?

If I choose the second activity of this course, I see this as an option.

Another possibility for a newly created activity might be: 

it might need to be saved once before the condition appears, --or--

the completion condition might need to be added to the activity and saved before the condition appears, --or--
maybe you can save changes, log out and log in again and then it becomes visible.

Doing all of the above definitely seems to work.

H5P on moodle - core activity vs mod_hvp plugin

Since mod_hvp plugin is not yet updated for Moodle version 4.5, we have deferred our upgrade. Looking at options, whether to use the built-in H5P support in Moodle, there's a discussion at 

https://moodle.org/mod/forum/discuss.php?d=444525

and the auto-translated German forum discussion is at

https://moodlemootdach-org.translate.goog/mod/forum/discuss.php?d=206&_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US&_x_tr_pto=wapp#p653

The main points seem to be: the built-in plugin does not have H5P Hub support, and also is "low priority" in Moodle development. But then, some Moodle developers say that is not the case, as in this discussion - 

https://moodle.org/mod/forum/discuss.php?d=443220

There is a migration tool, 

https://moodle.org/plugins/tool_migratehvp2h5p?lang=en

But then it seems to have some open issues like

https://github.com/moodlehq/moodle-tool_migratehvp2h5p/issues/24

So, it may be better to maintain status quo and wait for the mod_hvp plugin to be updated in a few months.

Moodle - change course backup location

 Exploring if we could change the "manual backup" location for courses, to put the backups in another directory rather than along with all the other files in moodledata/filedir, tried changing in Site Admin --> Courses --> Backups --> Automated backup setup -

But it looks like the path set there in backup_auto_destination seems to be valid only for automated backups. The manual backups initiated from the course are still saved in filedir. So, while taking backups of the entire server's moodledata directory, we are forced to take backups of backups.

Moodle plugin upgrade - click to upgrade not available

On one of our newly migrated Moodle instances, the "Click to update" button to update plugins for which updates are available, was not seen - only the button allowing us to download and then manually update the plugin was visible. As mentioned in this discussion,

https://moodle.org/mod/forum/discuss.php?d=434830

it was a permissions issue.

sudo chown -R azureuser:www-data .
sudo chmod -R 775 .

and hey presto - the install button came back when I went back to the plugins overview page.

rclone authentication by tunneling port did not work the second time

On one server, we used the method
https://hnsws.blogspot.com/2023/01/backing-up-mysql-databases-to-google.html
to set up an rclone remote with a shared google drive.

sudo dpkg -i rclone-current-linux-amd64.deb
to install latest rclone, tunnel port 53682 with putty so that we can authorize with browser.

When we tried with another server, exactly the same way, error that localhost refused to connect. That is, the port 53682 tunneling and authentication for rclone was not happening. 

Tried things like rebooting the local Windows machine, trying another browser and so on. But the only thing which worked was to do the "headless" option in rclone - first downloaded the latest rclone exe file, overwrote the version in C:\Windows which I had used earlier, and followed the online prompts.
https://rclone.org/remote_setup/

Maybe this is due to some sort of caching? But persisting after reboots? No idea. More probable might be some typo in configuring the tunnel in putty, I guess. Or maybe the port was being tunneled elsewhere or something like that.

default behaviour for unattended-upgrades

Since we had used Ubuntu 24.04 minimal images for some new servers, we had to check if unattended upgrades were installed and enabled -

https://askubuntu.com/questions/916722/how-to-verify-that-unattended-upgrades-are-enabled

Basically we have to check if an auto-upgrades file is present in /etc/apt/apt.conf.d/ and check if the Allowed-Origins are appropriate. In the case of this Ubuntu 24.04 minimal Azure image, the allowed origins not commented out were:

       "${distro_id}:${distro_codename}";

        "${distro_id}:${distro_codename}-security";

        // Extended Security Maintenance; doesn't necessarily exist for

        // every release and this system may not have it installed, but if

        // available, the policy for updates is such that unattended-upgrades

        // should also install from here by default.

        "${distro_id}ESMApps:${distro_codename}-apps-security";

        "${distro_id}ESM:${distro_codename}-infra-security";

So, all is good.

optimizing mysql for Moodle

Following

https://docs.moodle.org/405/en/Performance_recommendations#MySQL_Performance

That page pointed to
https://dev.mysql.com/doc/refman/5.7/en/innodb-buffer-pool-resize.html

SELECT @@innodb_buffer_pool_size/1024/1024/1024;
+------------------------------------------+
| @@innodb_buffer_pool_size/1024/1024/1024 |
+------------------------------------------+
|                           0.125000000000 |
+------------------------------------------+
1 row in set (0.00 sec)

mysql> \q

Can increase at least to 2G.

https://stackoverflow.com/questions/19534144/how-to-set-global-innodb-buffer-pool-size
mysql -u root -p
SET GLOBAL innodb_buffer_pool_size=2G;
ERROR 1232 (42000): Incorrect argument type to variable 'innodb_buffer_pool_size'

https://convertlive.com/u/convert/gigabytes/to/bytes#2

SET GLOBAL innodb_buffer_pool_size=2147483648;
Query OK, 0 rows affected (0.01 sec)

Running mysqltuner
https://www.linode.com/docs/guides/how-to-optimize-mysql-performance-using-mysqltuner/

-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
    Check warning line(s) in /var/log/mysql/error.log file
    Check error line(s) in /var/log/mysql/error.log file
    Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=ON
    We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
             See https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_join_buffer_size
    Be careful, increasing innodb_redo_log_capacity means higher crash recovery mean time
Variables to adjust:
    skip-name-resolve=ON
    join_buffer_size (> 256.0K, or always use indexes with JOINs)
    table_definition_cache (2000) > 3098 or -1 (autosizing if supported)
    innodb_buffer_pool_size (>= 12.4G) if possible.
    innodb_redo_log_capacity should be (=512M) if possible, so InnoDB Redo log Capacity equals 25% of buffer pool size.
    innodb_buffer_pool_instances(=2)
    innodb_log_buffer_size (> 16M)

So, on our Ubuntu 24.04 server,

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

skip-name-resolve=ON is not found
    join_buffer_size (> 256.0K, or always use indexes with JOINs) is not found

Tried editing my.cnf directly and restarted.
But restart failed. Need to enter with spaces between = ?
innodb_buffer_pool_size = 2G
This change worked OK on restarting mysql.

innodb_buffer_pool_size         = 2G
#join_buffer_size               = 256.0K this crashes.
innodb_redo_log_capacity        = 512M
innodb_buffer_pool_instances    = 2
innodb_log_buffer_size          = 16M

The above worked. Not doing the other recommendations, as a mysql restart crashes.

Some more recommendations, which we have not done:

Consider using postgresql instead.

Optimize your tables weekly and after upgrading Moodle. It is good practice to also optimize your tables after performing a large data deletion exercise, e.g. at the end of your semester or academic year. This will ensure that index files are up to date. Backup your database first and then use:
mysql>CHECK TABLE mdl_tablename;
mysql>OPTIMIZE TABLE mdl_tablename;

    The common tables in Moodle to check are mdl_course_sections, mdl_forum_posts, mdl_log and mdl_sessions (if using dbsessions). Any errors need to be corrected using REPAIR TABLE (see the MySQL manual and this forum script).

    Maintain the key distribution. Every month or so it is a good idea to stop the mysql server and run these myisamchk commands.

#myisamchk -a -S /pathtomysql/data/moodledir/*.MYI

    Warning: You must stop the mysql database process (mysqld) before running any myisamchk command. If you do not, you risk data loss.

understanding 'load average' in Linux

When we run the top command in Linux, we see three numbers depicting "Load average". What are they? and how much should they be?

https://scoutapm.com/blog/understanding-load-averages

1. The three numbers are one-minute, five-minute and fifteen-minute averages.

2. For a server, the load average should be 0.7 or below for a single CPU.

3. We can do cat /proc/cpuinfo to find the number of CPU cores. For a 2-core machine, below 2*0.7=1.4 would be healthy.

reduce the brightness of an hdri background

 Turn down camera exposure / settings, adjust strength of background image

Cycles render engine - Outdoor HDRI too bright - Blender Stack Exchange

https://blender.stackexchange.com/questions/69605/how-can-i-create-a-silhouette-using-hdr-in-cycles/69615#69615

Can use two nodes for background also, one to light the scene and one for the camera to see.

https://blender.stackexchange.com/questions/113908/increase-brightness-of-hdr-to-light-the-scene-but-keeping-the-background-at-a-l

Explanation of "factor" at

Make realistic environment lighting in Blender (in 10 mins) (youtube.com)

run LLMs locally - howtos

 Via slashdot - 

'Forget ChatGPT: Why Researchers Now Run Small AIs On Their Laptops' 

Nature published an introduction to running an LLM locally, 

which links to

5 easy ways to run an LLM locally | InfoWorld

For good (close to real-time) performance, would need 64 GB or RAM etc. But if speed of response is not important, we can use swap to run larger models than RAM allows, https://ryanagibson.com/posts/run-llms-larger-than-ram/

render multiple cameras simultaneously in Blender

 I didn't try this out - Render Multiple Cameras Simultaneously in Blender! (youtube.com)

using stereoscopy -> multi view  in output properties.

fulldome render with eevee and cycles

 This is an old post on blendernation about eevee on Blender v2.8 - Rendering Stereoscopic 360 footage with EEVEE - https://www.blendernation.com/2020/08/25/rendering-stereoscopic-360-footage-with-eevee/

(doesn't work with Blender 4.2 for me)

Rendering fulldome images in Cycles on Blender - 

Double-click the camera icon under camera to make its properties show up. Choosing Panoramic - Equisolid gives me good results.

google street view to video of trip

I had an idea of using google street view imagery and converting it to a video using google apps script or something similar. Obviously, many people have thought of this earlier.

https://github.com/TeehanLax/Hyperlapse.js
from
https://www.geographyrealm.com/create-videos-using-google-streetview-hyperlapse/

And the way to make it work seems to be to add an API key,
https://github.com/TeehanLax/Hyperlapse.js/issues/40#issuecomment-642969953

And apparently the API key now needs a billable account.
Mentioned here,
https://github.com/jblsmith/street-view-movie-maker
and forked for python3 here,
https://github.com/vardhman/street-view-movie-maker/tree/python3-support

So, currently no plans to try it out.

Google Sheets charts "add a series to start visualising your data" error and solution

Generally, we just select a range in Google Sheets and choose Insert -> Chart to get a chart automatically created from that range. But while debugging this issue, I had imported data from a text file which I had saved as csv after find/replace of spaces with commas, but google would not create a chart, instead showing "add a series to start visualising your data" and when I would add a range, it would say improper format.

It turned out that Google sheets had treated the numbers as text. So, the solution was to select the relevant ranges, Format -> Number -> Scientific (and then change back to Number or Automatic if required) . Once that was done, the automatic creation of charts worked.

video texture UV unwrapping

 https://www.youtube.com/watch?v=E1COCnMUIhQ - Import Any Video Into Blender With Animated Image Texture or Video Texture

https://www.youtube.com/watch?v=ZqsDjCVBkiw - Blender 3.4X: How to use a video as a texture

This one gives how to easily get UV right. 

Choose front view -- unwrap -- project from view.

Edit - https://docs.blender.org/manual/en/latest/editors/uv/controls/snapping.html

Hold Ctrl to snap UV to edges? - finally did not use, as we wanted cubes and the video was 16x9, so snapping the video to edges would have distorted.

dvddecrypter on wine etc - did not work

 For making dvd visible on wine,

https://ubuntuforums.org/showthread.php?t=1946371

check the path which is getting auto added

https://forum.winehq.org/viewtopic.php?t=32891

first run winecfg to make sure correct drive is mapped AFTER putting in the DVD, then run the application. Also may need to check the .wine/dosdevices directory and verify that spurious drives are not linked there.

wine faq - https://gitlab.winehq.org/wine/wine/-/wikis/FAQ

winecfg - was showing ejected drives also. must use wine eject, I guess.

https://gitlab.winehq.org/wine/wine/-/wikis/FAQ#removable-media

also this error

https://forum.winehq.org/viewtopic.php?t=37602

But still,

I/O error D:

Request not supported.

Interpretation: Inquiry

Since this says changed to nt4 and it worked,

https://forum.winehq.org/viewtopic.php?t=8448

tried with winecfg. But only options are Win XP and above. NT4 is not an option.

Did not work with XP or Win7 or Win10.

H5P error on Moodle 4.1

An error was reported in an H5P module in one of our Moodle instances. Copy-pasting from my response - 

The error says, undefined function str_contains()

From this page:
https://stackoverflow.com/questions/66519169/call-to-undefined-function-str-contains-php

"str_contains() was introduced in PHP 8 and is not supported in lower versions."

 Currently we're running Moodle 4.1 LTS (Long Term Support) which runs on PHP 7.4. We have an upgrade plan to upgrade to Moodle 4.5 LTS in October or so, when we would also upgrade the PHP version as well as the MySQL database version, along with the server Linux kernel version. We're waiting for October so that the LTS version of Moodle as well as the Linux kernel will be out - https://moodledev.io/general/releases .

I've explained the rationale for the upgrade here,

https://hnsws.blogspot.com/2024/04/moodle-upgrade-path-41-to-45.html

After the upgrade, this issue should be sorted.