Friday, January 03, 2025

nvidia accelerated video encoding with ffmpeg

Following this page,

https://docs.nvidia.com/video-technologies/video-codec-sdk/12.0/ffmpeg-with-nvidia-gpu/index.html

the following encoding went at around 21 fps with the NVidia 1060 graphics card - 

ffmpeg -r30 -start_number -003875 -f image2 -i "OpenSpace%06d.png" -c:v h264_nvenc ../filename.mp4

Using h265_nvenc gave codec not found error, but hevc_nvenc worked. That went at around 23 fps. Without acceleration, when using the "lossless" parameter, it was taking several seconds per frame. 0.2 fps or so.

Sunday, December 29, 2024

appimage issue and solution

There was a problem running the v1.60 appimage of OCVvid2fulldome, created in mid-2020 on Ubuntu 18.04, when trying to run on Linux Mint 22 (based on Ubuntu 24.04). 

AppImage: symbol lookup error: /lib/x86_64-linux-gnu/libgio-2.0.so.0: undefined symbol: g_module_open_full

Probably this is related to

So, opened an issue. But found that simply creating an appimage with Ubuntu 20.04 solves the problem. 

Tuesday, December 24, 2024

exFAT vs FAT32

I thought that exFAT was just the new name for FAT32. But apparently exFAT does not have the 4 GB file size limit, and can be read by MacOS natively (though not by all devices which can read FAT32) - https://superuser.com/questions/440509/getting-around-the-fat32-4-gb-file-size-limit


notes on backing up Azure blob storage folder to local external hard drive

Copy-pasting from an email exchange - 

As the message says, a few thousand files did not download properly, there were errors. Most probably due to bad internet connection. Would need to retry again and again until all are downloaded. Or, use a better internet connection - for eg, I use home BSNL fiber (100 Mbps) when I need to download large files. Another option is Airtel 5G, for one-time downloads. 

When I downloaded on BSNL Fiber using Azure explorer, the download went at around 50 Mbps, 111 GB in 17605 sec or just below 5 hours.

Copying from one hard disk to another showed another possible reason for the previous download to fail - the external hard disk was corrupted, leading to file copy failure - repaired the file system and the file copy worked OK.

If needed again, can explore rclone on the RPi.

Monday, December 23, 2024

server error 500 on trying to reset password in Moodle

A Moodle admin reported internal server error 500 on trying to reset the password for another user. And confirmed that the same error occurs even when resetting his own account's password from the profile field. But the password reset via the login page works.

So, perhaps there is some setting in Moodle which prevents anyone from changing passwords except via the password reset page
or
maybe there's a bug
or
there's some issue which prevents updating of user records.

I could reset the password using the commandline,
https://docs.moodle.org/405/en/Administration_via_command_line#Reset_user_password

which in our case is

sudo -u www-data /usr/bin/php admin/cli/reset_password.php
# this prompts for the username and then for the new password. 

 

 

Saturday, December 21, 2024

Let's Encrypt certificate expiry emails

Not sure if there is any way to tell letsencrypt that these certs are no longer required - or maybe certbot delete will do it?

certbot delete --cert-name mywebsite.com


Anyway, in this case, I have already de-allocated the old server, so will just ignore the emails after checking that the current certificates have a validity date beyond what is specified in the emails. 

capturing the screen in 4K resolution without a 4K monitor - with an NVidia graphics card on Linux

There's a very pretty visualization of all life on Earth - tree of life - on onezoom.org and I wanted to capture videos of zooming in and out.


Next, I wanted to see if I can do a 4K capture without needing to change the existing monitor, which runs on 1920x1080

Exploring the "NVidia X Server Settings", found a ViewPortIn setting after hitting the Advanced button under X Server Display Configuration. After setting it to 3840x2160, gpu-screen-recorder could still capture the screen at 60fps without any fuss. 


The Apply button worked, but the Save to X Configuration File did not - most probably due to some difference in the path. But an issue faced after making this change was that after every boot, the display would come back to this 4K setting, which made everything on the HD display too small for daily use. Even after changing back the ViewPortIn to HD, the Panning to HD, and Apply, the change would not remain after rebooting. The solution, which persisted across reboots, seemed to be to change the Scale from 2x to 1x in Display settings - 



Monday, December 16, 2024

outages of our Moodle instances

There were a couple of outages - at around 8 am on 16th when the database server became unresponsive, and around 2.45 pm when the web server became unresponsive. Rebooting fixed the issues on both occasions. I believe these could have been due to a kernel update? 

Sunday, December 08, 2024

Azure AD graph retirement

 There was an email from Microsoft Azure, "Migrate Service Principals from the retiring Azure AD Graph APIs to Microsoft Graph" - clicking through, the recommendation only showed the id of the function or resource using the deprecated API, and did not provide any other information.

After an internet search, this faq showed how to identify it - 
https://learn.microsoft.com/en-us/graph/migrate-azure-ad-graph-faq#as-an-it-admin-how-do-i-identify-apps-in-my-tenant-that-use-azure-ad-graph

(It was not an application which I had created, it says the owner is "Power Virtual Agents Service" - so no action was taken. More info - https://techcommunity.microsoft.com/blog/azure-ai-services-blog/use-your-own-data-to-create-a-power-virtual-agent-with-azure-openai-service/3891860 )

automating user preferences modifications on Moodle - SikuliX

We had to change a large number of users' forum preferences in Moodle as noted in the previous post. Checking the database for changes when such changes were made, it appeared that new entries would be added to prefix_user_preferences table, with 
name = message_provider_mod_forum_digests_enabled
value = popup
and so on.

Since this was not as straightforward as just changing a flag in a user preferences table, I thought it would be safer to do it via the Moodle UI instead of messing around with database changes. 

To automate the task of changing 200+ users' preferences, I once again took the help of SikuliX. Once again, I chose the simplistic method of making SikuliX scripts with hard-coded Location(x,y) values. Using the 'Run in slow motion' mode, which shows the Location with a red cross-hair, I used moveMouse commands to find the x and y co-ordinates of the points where I wanted the clicks to happen. Unfortunately, the points were not directly 1:1 corresponding to x and y co-ordinates of my 1920x1080 fullscreen capture - Locations based on those co-ordinates threw up errors that the co-ordinates did not correspond to any location (being out of range). 

With the Edge browser screen set to approx. 80% zoom in order to show all the elements we needed on the single screen, the two scripts to update the Moodle preferences pages were as follows.

Documents > notifpref.sikuli > notifpref.py

from time import sleep

sleeptime=0.5
sleeptillpageload=2.0
urlbar=Location(925,64)
test1=Location(1300,746)
test2=Location(1405,564)
webpref=Location(1180,746)
emailpref=Location(1300,746)
id=126

while (id<345): 
  click(urlbar)
  sleep(sleeptime)
  sleep(sleeptime)
  click(urlbar)
  #type(BACKSPACE 3 times)
  type("\b")
  type("\b")
  type("\b")
  type(str(id))
  #type ENTER
  type("\n")
  #mouseMove(test1)
  popup("waiting for no error")
  #sleep(sleeptime)
  #mouseMove(test2)
  mouseMove(webpref)
  #sleep(sleeptime)
  click(webpref)
  #mouseMove(emailpref)
  #sleep(sleeptime)
  click(emailpref)
  id=id+1
  #popup("waiting for no error")

Documents > forumprefs.sikuli > forumprefs.py

from time import sleep

sleeptime=0.5
sleeptillpageload=2.0
urlbar=Location(925,64)
subjectonly=Location(700,525)
test2=Location(505,704)
savebutton=Location(505,704)
emailtype=Location(700,446)
id=4362
url="https://ourserver.org/user/forum.php?id="

while (id>4135): 
  click(urlbar)
  #select all
  type("a",KeyModifier.CTRL)
  #type(paste the url already copied)
  type("v",KeyModifier.CTRL)
  type(str(id))
  #type ENTER
  type("\n")
  popup("waiting for no error")
  click(emailtype)
  sleep(sleeptime)
  click(subjectonly)
  id=id-1
  mouseMove(test2)
  click(savebutton)
  popup("waiting for no error")

  



  

Moodle database server overloaded - fixes

Since I'd set up CPU and disk space monitoring emails on some of our servers, I started getting CPU usage alert emails regularly from the mysql database server VM which served some of our Moodle instances. CPU usage kept going up, not declining even during night hours. 

Troubleshooting - 

  1. Checked 'currently running tasks' in all Moodle instances - Site administration > Server > Tasks > Tasks running now. Deleted the 3 tasks seen to be running for over an hour (delete modules) in the database table - since these were adhoc tasks, from the {task_adhoc} table based on adhoc taskid.

  2. Checked task logs, filtered for tasks taking longer than 3 seconds, and changed the scheduled tasks settings inside those Moodle instances to reduce the frequency of those tasks. Did this for all five Moodle instances. On the newest instance, changed the following in
    Site Administration > Server > Tasks > Scheduled tasks
    • \enrol_category\task\enrol_category_sync -  */5 instead of * (once every five minutes instead of every minute)

    • \local_edwiserreports\task\send_scheduled_emails 05 instead of */5 (once an hour instead of once a minute)

    • \local_edwiserreports\task\update_course_progress_data 39 instead of */5

    •  \mod_customcert\task\issue_certificates_task (once a day instead of every minute)

    •  \mod_forum\task\cron_task (once an hour instead of every minute)
    • \core\task\search_index_task */47 instead of */30 

  3. The newest instance had thousands of errors being generated by mod_forum - failed to send notifications - and also another instance had thousands of errors from 'failed to send login notifications' at the time of migration in May. Deleted all these failed adhoc tasks from prefix_task_adhoc for both these instances. (For deleting the rows, since the number of rows were so large, it was much faster to create a query like 
    delete from our_db.prefix_task_adhoc where classname like '%send_user_notif%' (24k rows)
    instead of deleting via the DBeaver GUI.

  4.  I had tried various options to prevent mod_forum from throwing errors for 'failed to send notifications'. Finally, the options which worked seemed to be:
    + Enable mobile notifications in Site Administration > General > Notification settings

    + Enable Subscribed forum posts notifications for Web and Mobile, also in Site Administration > General > Notification settings, default notification preferences

    + Enable Subscribed forum digests notifications for Web, Mobile and Email

    + Go to each user and change their forum notification preferences to Digest instead of No digest (each post separate email) - I'll write a separate post on how I automated this - Site Administration > Users , find the user, click on the user profile, Administration > Preferences and there, change forum preferences and notification preferences as above. 


GMail rate limiting and Moodle

Some of our Moodle instances which used XOAUTH to send emails via smtp.gmail.com had issues with temporary authentication failures. Apparently GMail's smtp servers have started rate limiting after approx 100 emails were sent in 3 minutes by Moodle. Mostly mod_forum email digests.

Since Moodle itself doesn't seem to have any provision for rate limiting, we need to set up some mail relay which can retry mails which get throttled.  

SSMTP which is currently installed on the server, doesn't seem to support any advanced features like rate limiting. 

Since I'm familiar with Postfix, looked up ways to send emails from Postfix through google's smtp servers - https://computingforgeeks.com/configure-postfix-to-relay-emails-using-gmail-smtp/

After setting up postfix as above, and changing the outgoing mail configuration on our Moodle instances to the default values (which would use the server default, ie postfix), emails seem to be going out fine. 

For checking the postfix logs for errors, 

journalctl -t postfix/smtp | more
journalctl -t postfix/smtp -f # for tailing the logs

With the postfix config with a single relayhost, there are a few errors once in a while "Network unreachable" but a second later, the mail gets sent. So, use of postfwd or extra config was not needed. If needed in future, multiple postfix instances or the use of multiple relay hosts based on authentication might be options.

Currently sending two large instances' emails through these, 250 emails have gone out last night with no problems as seen via the gmail interface in the sent folder.

 

limiting download speeds with trickle

Running Sheepit render farm in the background on one of our machines, I wanted to limit the download speed of the Sheepit client, since office bandwidth was limited. 

First tried wondershaper and tc - did not work - finally ended up with trickle. Modified the desktop shortcut launcher I use to the command

trickle -s -d 1024 -u 1024 java -jar sheepit-client-ver.jar

Earlier tries and misses, and links for reference - 

https://securitynetworkinglinux.com/how-to-shape-traffic-using-wondershaper-on-ubuntu-20-04-cli/

https://askubuntu.com/questions/1523362/server-24-04-only-starts-a-single-interface-on-boot

https://superuser.com/questions/1053003/what-is-the-difference-between-eth1-and-eno1
We have eno1 instead of eth0.

Used git clone instead of the apt version as per
https://github.com/magnific0/wondershaper

Then, syntax has changed. But errors,
sudo ./wondershaper -a eno1 -d 1024
Error: Exclusivity flag on, cannot modify.
Error: Exclusivity flag on, cannot modify.
Error: Exclusivity flag on, cannot modify.
RTNETLINK answers: File exists

Trying tc with this syntax from
https://superuser.com/questions/1598721/adding-both-delay-bandwidth-restrictions-via-tc

sudo tc qdisc add dev eno1 root netem rate 10mbit
no errors shown
sudo tc qdisc add dev eno1 root netem rate 1mbit
Error: Exclusivity flag on, cannot modify.

The 2nd command above was with protonvpn on.

sudo systemctl restart systemd-networkd
sudo tc qdisc add dev eno1 root netem rate 1mbit
Error: Exclusivity flag on, cannot modify.
 
https://www.cyberciti.biz/faq/linux-restart-network-interface/
sudo ifdown eno1
ifdown: command not found.

sudo systemctl restart systemd-networkd
 
sudo tc qdisc add dev eno1 root netem rate 5mbit
no errors,
but when running protonvpn, this has no effect - download is running at 2.4MB/s
 
trickle -s -d 1024 firefox
Command 'trickle' not found, but can be installed with:
sudo apt install trickle

sudo apt install trickle
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package trickle is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'trickle' has no installation candidate


https://ubuntu.pkgs.org/22.04/ubuntu-universe-amd64/trickle_1.07-11_amd64.deb.html
 
wget http://archive.ubuntu.com/ubuntu/pool/universe/t/trickle/trickle_1.07-11_amd64.deb
Installed OK.

But man trickle says,
Furthermore, trickle will not work with statically linked executables, nor with setuid(2) executables.
 
Also available from source, https://github.com/mariusae/trickle