satellite images of our solar panels

 Pretty pictures of our solar panel installation,  

https://livingatlas.arcgis.com/wayback/#active=9892&mapCenter=77.81341%2C14.16102%2C18

(without)

and

https://livingatlas.arcgis.com/wayback/#active=11351&mapCenter=77.81341%2C14.16102%2C18

(with)

appimage issue and solution

There was a problem running the v1.60 appimage of OCVvid2fulldome, created in mid-2020 on Ubuntu 18.04, when trying to run on Linux Mint 22 (based on Ubuntu 24.04). 

AppImage: symbol lookup error: /lib/x86_64-linux-gnu/libgio-2.0.so.0: undefined symbol: g_module_open_full

Probably this is related to

https://github.com/AppImage/AppImageKit/issues/1162

- a solution is mentioned at 
https://github.com/probonopd/linuxdeployqt/issues/526#issuecomment-1911779101

So, opened an issue. But found that simply creating an appimage with Ubuntu 20.04 solves the problem. 

'verify your account' emails from google cloud platform

We keep receiving emails from CloudPlatform-noreply@google.com asking us to verify "your google cloud account" followed by a number. But clicking on the Verify button leads to a "not found" page on google cloud console. Clicking on the "Support" link at the bottom of the email, I came to the link for "interactive billing assistant" at

https://cloud.google.com/support/billing

which led to

https://apps.google.com/supportwidget/helphome?product_name=Cloud+Platform&product_context=billing/chat&hl=en_US

That "AI assistant" suggested that this may be a message to verify the billing account payment method. I see that a billing account with payment method had been created in the past, but we're not currently using any paid services. 

The accounts dept verified the payment method with credit card, but we still get these messages. Probably due to some other account which was used to create google assistant action? Perhaps.

exFAT vs FAT32

I thought that exFAT was just the new name for FAT32. But apparently exFAT does not have the 4 GB file size limit, and can be read by MacOS natively (though not by all devices which can read FAT32) - https://superuser.com/questions/440509/getting-around-the-fat32-4-gb-file-size-limit

notes on backing up Azure blob storage folder to local external hard drive

Copy-pasting from an email exchange - 

As the message says, a few thousand files did not download properly, there were errors. Most probably due to bad internet connection. Would need to retry again and again until all are downloaded. Or, use a better internet connection - for eg, I use home BSNL fiber (100 Mbps) when I need to download large files. Another option is Airtel 5G, for one-time downloads. 

When I downloaded on BSNL Fiber using Azure explorer, the download went at around 50 Mbps, 111 GB in 17605 sec or just below 5 hours.

Copying from one hard disk to another showed another possible reason for the previous download to fail - the external hard disk was corrupted, leading to file copy failure - repaired the file system and the file copy worked OK.

If needed again, can explore rclone on the RPi.

server error 500 on trying to reset password in Moodle

A Moodle admin reported internal server error 500 on trying to reset the password for another user. And confirmed that the same error occurs even when resetting his own account's password from the profile field. But the password reset via the login page works.

So, perhaps there is some setting in Moodle which prevents anyone from changing passwords except via the password reset page
or
maybe there's a bug
or
there's some issue which prevents updating of user records.

I could reset the password using the commandline,
https://docs.moodle.org/405/en/Administration_via_command_line#Reset_user_password

which in our case is

sudo -u www-data /usr/bin/php admin/cli/reset_password.php

# this prompts for the username and then for the new password. 

 

 

Let's Encrypt certificate expiry emails

Not sure if there is any way to tell letsencrypt that these certs are no longer required - or maybe certbot delete will do it?

certbot delete --cert-name mywebsite.com

https://stackoverflow.com/questions/40030537/how-to-stop-renewing-a-letsencrypt-certbot-certificate

Anyway, in this case, I have already de-allocated the old server, so will just ignore the emails after checking that the current certificates have a validity date beyond what is specified in the emails. 

chroma keying tutorial etc - for "one zoom" masking of white

In order to project the onezoom.org video on the dome, without all the white background dazzling viewers, looked for ways to convert the white background to black background.

Chromakeyer tutorial - https://www.youtube.com/watch?v=dI1kMT3O6uk

But we might not be able to use that? Maybe if we make the clip green?

Green screen tutorial - Ryan King Art

https://www.youtube.com/watch?v=KBBkgoDPnZE

Finally ended up using inverted Y images (video) as mask, then increased contrast 3x with Multiply 3x.

capturing the screen in 4K resolution without a 4K monitor - with an NVidia graphics card on Linux

There's a very pretty visualization of all life on Earth - tree of life - on onezoom.org and I wanted to capture videos of zooming in and out.

Via https://old.reddit.com/r/linux_gaming/comments/tjkvyd/i_made_a_nvidia_shadowplaylike_screen_recorder/ - gpu-screen-recorder

Next, I wanted to see if I can do a 4K capture without needing to change the existing monitor, which runs on 1920x1080

Exploring the "NVidia X Server Settings", found a ViewPortIn setting after hitting the Advanced button under X Server Display Configuration. After setting it to 3840x2160, gpu-screen-recorder could still capture the screen at 60fps without any fuss. 

The Apply button worked, but the Save to X Configuration File did not - most probably due to some difference in the path. But an issue faced after making this change was that after every boot, the display would come back to this 4K setting, which made everything on the HD display too small for daily use. Even after changing back the ViewPortIn to HD, the Panning to HD, and Apply, the change would not remain after rebooting. The solution, which persisted across reboots, seemed to be to change the Scale from 2x to 1x in Display settings - 

preliminary work - tree of life from onezoom.org

 Onezoom.org has this beautiful visualization of the "tree of life" - zooming in and out of the "homo sapiens" entry - captured using GPU Screen Recorder at 60 fps, and slowed down to 15 fps without any motion compensation etc using avidemux.

The original video is also shared on archive.org

outages of our Moodle instances

There were a couple of outages - at around 8 am on 16th when the database server became unresponsive, and around 2.45 pm when the web server became unresponsive. Rebooting fixed the issues on both occasions. I believe these could have been due to a kernel update? 

Azure AD graph retirement

 There was an email from Microsoft Azure, "Migrate Service Principals from the retiring Azure AD Graph APIs to Microsoft Graph" - clicking through, the recommendation only showed the id of the function or resource using the deprecated API, and did not provide any other information.

After an internet search, this faq showed how to identify it - 
https://learn.microsoft.com/en-us/graph/migrate-azure-ad-graph-faq#as-an-it-admin-how-do-i-identify-apps-in-my-tenant-that-use-azure-ad-graph

(It was not an application which I had created, it says the owner is "Power Virtual Agents Service" - so no action was taken. More info - https://techcommunity.microsoft.com/blog/azure-ai-services-blog/use-your-own-data-to-create-a-power-virtual-agent-with-azure-openai-service/3891860 )

automating user preferences modifications on Moodle - SikuliX

We had to change a large number of users' forum preferences in Moodle as noted in the previous post. Checking the database for changes when such changes were made, it appeared that new entries would be added to prefix_user_preferences table, with 

name = message_provider_mod_forum_digests_enabled

value = popup
and so on.

Since this was not as straightforward as just changing a flag in a user preferences table, I thought it would be safer to do it via the Moodle UI instead of messing around with database changes. 

To automate the task of changing 200+ users' preferences, I once again took the help of SikuliX. Once again, I chose the simplistic method of making SikuliX scripts with hard-coded Location(x,y) values. Using the 'Run in slow motion' mode, which shows the Location with a red cross-hair, I used moveMouse commands to find the x and y co-ordinates of the points where I wanted the clicks to happen. Unfortunately, the points were not directly 1:1 corresponding to x and y co-ordinates of my 1920x1080 fullscreen capture - Locations based on those co-ordinates threw up errors that the co-ordinates did not correspond to any location (being out of range). 

With the Edge browser screen set to approx. 80% zoom in order to show all the elements we needed on the single screen, the two scripts to update the Moodle preferences pages were as follows.

Documents > notifpref.sikuli > notifpref.py

from time import sleep

sleeptime=0.5

sleeptillpageload=2.0

urlbar=Location(925,64)

test1=Location(1300,746)

test2=Location(1405,564)

webpref=Location(1180,746)

emailpref=Location(1300,746)

id=126

while (id<345): 

  click(urlbar)

  sleep(sleeptime)

  sleep(sleeptime)

  click(urlbar)

  #type(BACKSPACE 3 times)

  # https://answers.launchpad.net/sikuli/+faq/1444

  type("\b")

  type("\b")

  type("\b")

  type(str(id))

  #type ENTER

  type("\n")

  #mouseMove(test1)

  popup("waiting for no error")

  #sleep(sleeptime)

  #mouseMove(test2)

  mouseMove(webpref)

  #sleep(sleeptime)

  click(webpref)

  #mouseMove(emailpref)

  #sleep(sleeptime)

  click(emailpref)

  id=id+1

  #popup("waiting for no error")

Documents > forumprefs.sikuli > forumprefs.py

from time import sleep

sleeptime=0.5

sleeptillpageload=2.0

urlbar=Location(925,64)

subjectonly=Location(700,525)

test2=Location(505,704)

savebutton=Location(505,704)

emailtype=Location(700,446)

id=4362

url="https://ourserver.org/user/forum.php?id="

while (id>4135): 

  click(urlbar)

  #select all

  type("a",KeyModifier.CTRL)

  #type(paste the url already copied)

  type("v",KeyModifier.CTRL)

  type(str(id))

  #type ENTER

  type("\n")

  popup("waiting for no error")

  click(emailtype)

  sleep(sleeptime)

  click(subjectonly)

  id=id-1

  mouseMove(test2)

  click(savebutton)

  popup("waiting for no error")

  

  

Moodle database server overloaded - fixes

Since I'd set up CPU and disk space monitoring emails on some of our servers, I started getting CPU usage alert emails regularly from the mysql database server VM which served some of our Moodle instances. CPU usage kept going up, not declining even during night hours. 

Troubleshooting - 

1. Checked 'currently running tasks' in all Moodle instances - Site administration > Server > Tasks > Tasks running now. Deleted the 3 tasks seen to be running for over an hour (delete modules) in the database table - since these were adhoc tasks, from the {task_adhoc} table based on adhoc taskid.
   
   
2. Checked task logs, filtered for tasks taking longer than 3 seconds, and changed the scheduled tasks settings inside those Moodle instances to reduce the frequency of those tasks. Did this for all five Moodle instances. On the newest instance, changed the following in
   Site Administration > Server > Tasks > Scheduled tasks
   
   • \enrol_category\task\enrol_category_sync -  */5 instead of * (once every five minutes instead of every minute)
     
     
   • \local_edwiserreports\task\send_scheduled_emails 05 instead of */5 (once an hour instead of once a minute)
     
     
   • \local_edwiserreports\task\update_course_progress_data 39 instead of */5
     
     
   •  \mod_customcert\task\issue_certificates_task (once a day instead of every minute)
     
     
   •  \mod_forum\task\cron_task (once an hour instead of every minute)
   • \core\task\search_index_task */47 instead of */30 
     
     
3. The newest instance had thousands of errors being generated by mod_forum - failed to send notifications - and also another instance had thousands of errors from 'failed to send login notifications' at the time of migration in May. Deleted all these failed adhoc tasks from prefix_task_adhoc for both these instances. (For deleting the rows, since the number of rows were so large, it was much faster to create a query like 
   delete from our_db.prefix_task_adhoc where classname like '%send_user_notif%' (24k rows)
   instead of deleting via the DBeaver GUI.
   
   
4.  I had tried various options to prevent mod_forum from throwing errors for 'failed to send notifications'. Finally, the options which worked seemed to be:
   + Enable mobile notifications in Site Administration > General > Notification settings
   
   + Enable Subscribed forum posts notifications for Web and Mobile, also in Site Administration > General > Notification settings, default notification preferences
   
   + Enable Subscribed forum digests notifications for Web, Mobile and Email
   
   + Go to each user and change their forum notification preferences to Digest instead of No digest (each post separate email) - I'll write a separate post on how I automated this - Site Administration > Users , find the user, click on the user profile, Administration > Preferences and there, change forum preferences and notification preferences as above. 
   
   
   

GMail rate limiting and Moodle

Some of our Moodle instances which used XOAUTH to send emails via smtp.gmail.com had issues with temporary authentication failures. Apparently GMail's smtp servers have started rate limiting after approx 100 emails were sent in 3 minutes by Moodle. Mostly mod_forum email digests.

Since Moodle itself doesn't seem to have any provision for rate limiting, we need to set up some mail relay which can retry mails which get throttled.  

SSMTP which is currently installed on the server, doesn't seem to support any advanced features like rate limiting. 

Since I'm familiar with Postfix, looked up ways to send emails from Postfix through google's smtp servers - https://computingforgeeks.com/configure-postfix-to-relay-emails-using-gmail-smtp/

After setting up postfix as above, and changing the outgoing mail configuration on our Moodle instances to the default values (which would use the server default, ie postfix), emails seem to be going out fine. 

For checking the postfix logs for errors, 

journalctl -t postfix/smtp | more
journalctl -t postfix/smtp -f # for tailing the logs

With the postfix config with a single relayhost, there are a few errors once in a while "Network unreachable" but a second later, the mail gets sent. So, use of postfwd or extra config was not needed. If needed in future, multiple postfix instances or the use of multiple relay hosts based on authentication might be options.

Currently sending two large instances' emails through these, 250 emails have gone out last night with no problems as seen via the gmail interface in the sent folder.

 

limiting download speeds with trickle

Running Sheepit render farm in the background on one of our machines, I wanted to limit the download speed of the Sheepit client, since office bandwidth was limited. 

First tried wondershaper and tc - did not work - finally ended up with trickle. Modified the desktop shortcut launcher I use to the command

trickle -s -d 1024 -u 1024 java -jar sheepit-client-ver.jar

Earlier tries and misses, and links for reference - 

https://securitynetworkinglinux.com/how-to-shape-traffic-using-wondershaper-on-ubuntu-20-04-cli/

https://askubuntu.com/questions/1523362/server-24-04-only-starts-a-single-interface-on-boot

https://superuser.com/questions/1053003/what-is-the-difference-between-eth1-and-eno1
We have eno1 instead of eth0.

Used git clone instead of the apt version as per
https://github.com/magnific0/wondershaper

Then, syntax has changed. But errors,
sudo ./wondershaper -a eno1 -d 1024
Error: Exclusivity flag on, cannot modify.
Error: Exclusivity flag on, cannot modify.
Error: Exclusivity flag on, cannot modify.
RTNETLINK answers: File exists

Trying tc with this syntax from
https://superuser.com/questions/1598721/adding-both-delay-bandwidth-restrictions-via-tc

sudo tc qdisc add dev eno1 root netem rate 10mbit
no errors shown
sudo tc qdisc add dev eno1 root netem rate 1mbit
Error: Exclusivity flag on, cannot modify.

The 2nd command above was with protonvpn on.

sudo systemctl restart systemd-networkd

sudo tc qdisc add dev eno1 root netem rate 1mbit

Error: Exclusivity flag on, cannot modify.

 

https://www.cyberciti.biz/faq/linux-restart-network-interface/
sudo ifdown eno1
ifdown: command not found.

sudo systemctl restart systemd-networkd

 

sudo tc qdisc add dev eno1 root netem rate 5mbit
no errors,
but when running protonvpn, this has no effect - download is running at 2.4MB/s

 

trickle -s -d 1024 firefox
Command 'trickle' not found, but can be installed with:
sudo apt install trickle

sudo apt install trickle
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package trickle is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'trickle' has no installation candidate

https://ubuntu.pkgs.org/22.04/ubuntu-universe-amd64/trickle_1.07-11_amd64.deb.html

 
wget http://archive.ubuntu.com/ubuntu/pool/universe/t/trickle/trickle_1.07-11_amd64.deb
Installed OK.

But man trickle says,

Furthermore, trickle will not work with statically linked executables, nor with setuid(2) executables.

 

Also available from source, https://github.com/mariusae/trickle

 

 

some blender ideas I have not yet tried

Shortcut to creating rocket scene - from https://sketchfab.com/3d-models/gslv-mk3-0426922358b4444f9887bcd551d3a5cb - zoom in the view, take screenshots, then composite it with Earth background.

Editing Blender Image Texture with Gimp - How to Link Blender with any External Image Editor! (youtube.com) - as of now, only directly edited the png texture file(s) by loading them into Gimp outside Blender.

How to make millions of bodies - https://www.youtube.com/watch?v=CQ9VmCN2EsE - How to Render Millions of Objects in Blender.

Rigid bodies physics - falling etc https://www.youtube.com/watch?v=nHVYYMG3QVY

DNA in Blender in one minute - nicely explained - https://www.youtube.com/watch?v=xgPlgiOQWPA

Biochemistry L12: Building Bacteria (E. coli) in Blender - https://www.youtube.com/watch?v=XoDDNCWyziI - that channel has lots of Molecules, Chemistry, proteins, petri-dish etc Blender tutorials. @LuminousLab, "Blender for Scientists".

Smart UV project - for easier uv unwrapping of simple objects - https://www.youtube.com/watch?v=qa_1LjeWsJg - UV > Smart UV project - https://youtu.be/qa_1LjeWsJg?feature=shared&t=820 - This did not seem to work for the cubes - would always turn out sideways for some cube faces. Did rotation and rescaling of the uv mesh manually.

Idea for Swami video composition - or a sequence of stills - 5 stills left to right, come at 1 sec intervals, 4 stills above that (found in testing that one layer of stills looks better than 2 layers.) Each frame changes every second? May not be required - could have new frames appearing every second, then after 5 seconds, the first frame changes, etc. Change would be like a page turn? - or not - quick fade to black and change also works. Another option: A large photo album, with pages turning. Each page has 2 + 2 photos? This might need animation. So, earlier option might be easier. Page turn tutorial - https://www.youtube.com/watch?v=K3lfNXAZblA

Keyframe the multiply factor, to change the speed of clips - to slowly speed up or slow down clips - https://docs.blender.org/manual/en/latest/video_editing/edit/montage/strips/effects/speed_control.html - as of now, used only the "stretch" method, which is the default for the Speed Control effect, which is easy to use. We can cut clips with Shift-K and apply different speeds etc.

Privacy blur mask in Blender VSE - https://www.youtube.com/watch?v=v0qoIRKNtnE - 

Set active camera - animate camera smoothly - https://www.youtube.com/watch?v=a7qyW1G350g

How to use google colab to render blender files - Speed Up Your Blender Renders with Free Google Colab- using Sheepit Render Farm instead as of now.

3d text in Blender - Add a text object, choose the font in the object's properties. 3D Text in Blender: Everything You Need to Know! - but has problems with Hindi rendering etc

human body anatomy - using the free z-anatomy blender model

Led on by this video, though the model was not found in the link in the description of the video, found it with a github search at Z-Anatomy/The-blend: Z-Anatomy blender template

As mentioned in the video, importing single collections like skeletal system takes a minute or so, importing multiple collections or opening the entire blend file takes a bit longer.

To import the whole thing and then save-as, deleting those features which we don't want - 

• to delete text labels, we can select all text labels by going to Object menu, Select all by type and choosing Text
  
  
• Rendering in eevee or cycles causes a cut-away view, while workbench rendering engine gives a whole face. This is probably due to some "hide" setting somewhere, but I just went with workbench rendering instead of trying to troubleshoot.
  
  
• to turn on visibility for all objects in a collection, we have to Alt-left-click on the render / viewport icon as required. ALT clicking is the trick.
  
  
• to move camera to current view, the shortcut is Ctrl+Alt+Numpad 0. Or can change the shortcut on machines which don't have the numpad, in File > User Preferences > Input, search for camera, the pref is 'align camera to view'. For Blender 4.2, the menu is Edit > Preferences > Keymap
  
  
• to prevent camera being deleted with lasso select, just hide it - select the object in object mode, Object menu > Show/Hide > Hide selected (or H is the shortcut key, Shift H to unhide after the delete operations are done.)
  
  
• Working with objects inside a collection - simply selecting the collection and deleting it does not delete the objects inside the collection. To select all objects in a collection - right-click the collection and choose Select objects. Then, Del key or x or object menu > delete