scheduled start and stop for VMs on Azure

Setting up auto-shutdown on Azure VMs is easy, there is an option under Operations - Auto-shutdown directly in the VM's left-hand side panel. Setting up auto-startup is more involved. Currently, Azure has a Start/Stop VMs V2 feature which can directly be deployed. But since it makes use of some storage and so on, it is billed separately. And it is not available to deploy in the "South India" region. But still, choosing the default region, it is able to start our "South India" VM when I configured it using the "start all VMs in a particular Resource group" method, like

"RequestScopes": { "ResourceGroups": [ "/subscriptions/12our78-1sub4-56id8-1234-123456781234/resourceGroups/OurResourceGrp/" ],

as in the example linked above.

dynamic dns options

I wanted to try out self-hosting using BSNL's fiber internet. One of the ways to get dynamic dns to work is to use Cloudflare's api and the script given at

https://adamtheautomator.com/cloudflare-dynamic-dns/

Unfortunately the script needs Powershell 7, which only ships with Windows Server - I was trying out on a Win11 home machine. Even with Win10 Pro, this approach may not work. 

Of course, there are bash scripts which can do the job, like

https://gist.github.com/amiller-gh/7d7dcb4176ee9f42f13e4fe672d4a7f5

But, since it was available, tried the Dynamic DNS option in our syRoTech ONT - Device Model SY-GPON-1110-WDONT. 

Under Application - DDNS menu option, we can set dynamic dns from oray.com (a Chinese site), DynDNS and NoIP. Since NoIP has a free plan, set up a free hopto.org sub-domain with NoIP, and made appropriate CNAMEs in our organisation's DNS - working fine. 

youtube hosted videos on Moodle

There was a query from one of our units, asking how to disable the "Share" link to youtube videos which they embed in their Moodle course. Copy-pasting from my reply:

Some possibilities -

Option 1 - Youtube Custom player embed -
https://www.lifewire.com/embedded-youtube-video-customization-1082489
(I've not tried it, so I don't know if this will help.)

https://www.youtube.com/watch?v=f0iN7jfM7vs

The above video says that you can just add two parameters like controls=0, but now youtube no longer supports the showinfo=0 parameter.

Option 2 - Custom javascript player embed - something like
https://tikku.com/open-source/jquery-plugins/youtube-tubeplayer-plugin/
(I've not tried it, so I don't know if this will help.)

Option 3 - Youtube hosted videos have an option to make them "private".

Then, only those people who have been specifically given permission to view them can see them.

https://support.google.com/youtube/answer/157177?hl=en&co=GENIE.Platform%3DDesktop#zippy=%2Cprivate-videos

Disadvantage - The user would need to be signed in to their google account in order to view the video.

Option 4 - Putting the videos elsewhere, like Google Drive or some other provider like vimeo,
https://www.colby.edu/acits/2020/01/30/how-to-embed-a-video-from-google-drive/

setting up DMARC for better email delivery, feedback loops and XML DMARC reports

Copy-pasting from some emails - 

Google Workspace (GSuite), which we use for sending our domain's emails, makes it quite easy to set up DMARC for better email delivery.

Setting up DMARC involves

1. setting up DKIM signing for all outbound emails from our domain (SPF is already set up)

2. setting up an email id (or group) which will receive all spam notifications 

If emails are being sent only from Google's servers and not using any other servers, the setup is relatively straight-forward. In case any other servers are being used, those also would need to support DKIM signing. 

But I was mistaken about the DMARC reports being useful for finding out which recipient's spam filter is blocking our emails - it was not DMARC which we enabled in 2009, it was Yahoo's complaint feedback loop, for which we needed to set up DKIM. The DMARC reports don't seem to give us detailed info about spam complaints, while the complaint loop did.

https://hnsws.blogspot.com/2009/05/signing-emails-with-dkim.html

Our emails being sent via Google Apps Script are to GMail users - Google does have a feedback loop mechanism, but it needs additional header information to work - https://support.google.com/mail/answer/6254652?hl=en

And Google Apps Script MailApp does not support DKIM signing with our domain.

Viewing the xml DMARC reports - to make them more human-readable, tried out XSLT from https://www.tana.it/sw/dmarc-xsl/#xslt - didn't seem to work. 

( For this, tried editing the xml file in the reports, and adding the line

<?xml-stylesheet type = "text/xsl" href = "dmarcstyle.xsl"?>

after saving the script above as the file dmarcstyle.xsl in in the same folder as the dmarc report xml file.

Then right-click and open the xml file in some browser. But didn't seem to work. Tutorial on how to use XSLT - 

https://www.youtube.com/watch?v=W--Yhp0m35A

)

 Then tried dmarcian's XML to Human converter, gives good results.

Google Apps Script importxml for xml file stored in Google Drive

This link - https://stackoverflow.com/questions/45988417/how-to-use-importxml-function-with-a-file-from-google-drive - gives a good overview of how to use a file stored in Google Drive in Google Apps script. Either it has to be shared as viewable by all, or it has to be read using DriveApp.

meaning of the tags in DKIM signature

When setting up DMARC, we have to set up DKIM first. Google Workspace makes it easy. But I was curious to know what exactly the tags mean in the DKIM header. From
https://help.returnpath.com/hc/en-us/articles/222438487-DKIM-signature-header-detail

v = version

a = algorithm of the hash

h = hash

s = selector record name in DNS

d = domain used with the selector s

b = hash data (body)

bh = computed hash of the message body

h = list of headers used in the hash algorithm

t = timestamp of the hash

x = expiry time of signature

c = canonicalization algorithm in case mail server makes minor changes

i = identity of user or agent

LetsEncrypt certificate renewal issue with Cloudflare proxied Bitnami server

Found one of the Bitnami servers on AWS had an expired LetsEncrypt certificate. Checking, found that its bncert-tool was not able to renew the certificate because it was checking DNS as part of validation, and the server was being proxied by Cloudflare. 

1. Temporary solution - turning proxying off (DNS only) in cloudflare, ran bncert to renew, OK for now.
2. Looking for ways in which the renewal can happen without us having to manually change the cloudflare settings, I found the following:
   https://nodehost.ca/docs/containers/make-lets-encrypt-work-when-using-cloudflare-acme-challenge
   
   But this is when the host uses certbot and not bitnami's bncert-tool, which uses a different authentication method. This does not work out of the box for Bitnami servers since the config files' paths are different. On the server itself, there is the rule
   RewriteCond %{REQUEST_URI} !^/\.well-known
   in the file /opt/bitnami/apache2/conf/bitnami/bitnami.conf
3. Possibly the final solution:
   bncert-tool uses the lego client, documentation at
   https://go-acme.github.io/lego/usage/cli/options/
   
   From the documentation, it looks like we can choose Cloudflare as the DNS provider and get the actual dns address. For this, we will need an api key for lms.sssihl.edu.in -
    via
   Creating API tokens · Cloudflare API docs
   
   (Using api key instead of api token for initial testing,
   Cloudflare and restricted API tokens · Issue #984 · go-acme/lego (github.com) )
   
   Then we can try and configure the server to auto update the ssl certificates by authenticating the IP address from cloudflare.
   
   So, I have added two files on the server,
   /home/bitnami/cloudflare-user
   and
   /home/bitnami/cloudflare-api-key
   
   and I have modified the cron command which runs daily to check for renewal,
   from
   51 23 * * * sudo /opt/bitnami/letsencrypt/lego --path /opt/bitnami/letsencrypt --email="theemail@thedomain.in" --http --http-timeout 30 --http.webroot /opt/bitnami/apps/letsencrypt --domains=thedomain.in --user-agent bitnami-bncert/1.0.0 renew && sudo /opt/bitnami/apache/bin/httpd -f /opt/bitnami/apache/conf/httpd.conf -k graceful # bncert-autorenew
   
   to
   51 23 * * * sudo CLOUDFLARE_EMAIL_FILE=/home/bitnami/cloudflare-user CLOUDFLARE_API_KEY_FILE=/home/bitnami/cloudflare-api-key /opt/bitnami/letsencrypt/lego --path /opt/bitnami/letsencrypt --dns cloudflare --email="theemail@thedomain.in" --http --http-timeout 30 --http.webroot /opt/bitnami/apps/letsencrypt --domains=thedomain.in --user-agent bitnami-bncert/1.0.0 renew && sudo /opt/bitnami/apache/bin/httpd -f /opt/bitnami/apache/conf/httpd.conf -k graceful # bncert-autorenew
   
   Let's see after 30 days if it successfully renews. Seemed to work OK during testing.
   Edit: 14 Sep - Yes, it has auto-renewed on 10 Sep.
   

Some other alternative solutions, which I did not explore:

1. There is an alternative method using self-signed certificates, but that would need cloudflare HTTPS setting to be changed to Full instead of Full (Strict), https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/ )
    
2. Another option is, you could create an Origin Certificate using Cloudflare, and use that instead of the LetsEncrypt certificate.
   https://developers.cloudflare.com/ssl/origin-configuration/origin-ca/
   This would need a one-time set up on the server, after that would not need renewal (for 15 years.)
   Setup on the server would be similar to this, https://visser.io/2018/03/setting-up-a-cloudflare-ssl-certificate-for-bitnami-wordpress-on-google-cloud-compute-engine/
   Edit - I have implemented this, and posted about it here.
   
3. There is probably another option using manual configuration of certbot, but more difficult than the above, so I'm skipping that.
   

accessibility toolbars and PDFs

There was a query about meeting WCAG standards.  

 Diksha.gov.in also meets WCAG standards.
All websites provide accessibility tool bar consisting of:
1) Font size increase of text
2) Dark mode or contrast.
Moodle version 4 also provides this if enabled.

On Diksha, it is found that both the tools do not work on PDFs and PPTs and only  work on other content on website. What could be the reason for this?

Copy-pasting from my reply, 

The toolbars usually used on websites generally use javascript to modify the display of HTML content by the web browser (like Chrome, Firefox, etc). PDF, PPT and other content are not displayed on the screen by the web browser directly, but instead depend on other software like PDF plugins, Microsoft Office etc. Therefore, the toolbars will only work with HTML content.

If the Userway widget works with PDF and PPT , it would be because they use some other technique for changing the display.

Windows boot error - A data disk is currently set as active in BIOS

An error encountered when trying A's Windows install -  "A data disk is currently set as active in BIOS. Set some other disk as active or use the DiskPart utility to clean the data disk, and then retry the restore operation. (0x80042406)"

https://answers.microsoft.com/en-us/windows/forum/all/a-data-disk-is-currently-set-as-active-in-bios-set/eb931d76-9d23-424c-ba40-8fcd20633125

Apparently this means that the boot partition is borked and we have to boot with a System Restore disk and then restore the image.

Windows 10 recovery - cannot be done with Win7 startup disk

There was a Windows 10 installation on a laptop which needed boot recovery after Linux had been installed on another partition and then removed or something like that. Tried with a Windows 7 DVD - the bootup disk complained that "this startup disk is not compatible with the Recovery version" etc - so I downloaded Win10 startup disk (install disk) from Microsoft, created a USB boot disk using Rufus, and used that to recover the Win10 installation.

Openshot vs Avidemux - ffmpeg - avisynth - Virtualdub

Checked out Openshot for cropping and resizing some videos. https://www.youtube.com/watch?v=uUyqWu8A2OQ

(Advanced view)
--> crop size
--> crop left
(location)

Much faster to do it in Avidemux. For my needs, Openshot does not seem to be a good fit. Avidemux - ffmpeg - avisynth - Virtualdub seem to be more suited for my requirements.

Google group subscriptions which don't list the email to group owners

Found that there were some Google group subscriptions which don't list the email to group owners also - when we search by email for members, these don't show up. But if we choose the filter "Not in Org" and in Descending order, such subscribers are listed by name at the top. We could then see if the displayed name is similar to the subscriber we are looking for.

Windows 10 network time update fails on BSNL FTTH - NetTime works

BSNL's FTTH seems to block UDP port 123, hence Windows 10 network time update fails as noted in this set of posts, which also gives the workaround, installing NetTime.

Windows time sync issue with BSNL FTTH and workaround

Noticed that the Windows 11 machine I was working on had 11:32 showing as the time when it was well into the afternoon. Found this thread,
https://broadbandforum.co/threads/cannot-sync-network-time-ntp-on-win10-on-bsnl-ftth.202104/

So apparently Windows tries to poll port 123 from port 123, and that is blocked by many ISPs. 

The easiest workaround was to install NetTime which probably changes the source port and gets around the block. 

sending mail from GMail smtp servers instead of postfix

Our internal server used to send us notification emails for various server tasks. Recently, our server started getting blocked by GMail with 550-5.7.1 "likely unsolicited mail". Since setting up a "proper" smtp server with reverse DNS, DKIM signing, etc etc would be more painful than just using GMail SMTP servers, we preferred using one of the solutions below - 

https://netcorecloud.com/tutorials/send-an-email-via-gmail-smtp-server-using-php/

or

https://netcorecloud.com/tutorials/linux-send-mail-from-command-line-using-smtp-server/

Since our server scripts were bash scripts and not PHP scripts, we used the latter. 

sudo apt install ssmtp
(which would remove postfix)
and modifying the configuration file as in the link above, /etc/ssmtp/ssmtp.conf with

mailhub=smtp.gmail.com:587

AuthUser=ouruser@ourdomain.org

AuthPass=ourpassword

useSTARTTLS=YES

useTLS=YES

(and allow less secure apps must be enabled on that account.)

With that, the notification emails from our server resumed. 

changes after upgrading to Moodle 4

Copy-pasting edited extracts from an email exchange after upgrading a test server to Moodle 4 - 

... the orange band appears with company logo. Need to check how to remove orange band or replace the logo with SSSVV.

 

Also the footer information is not displayed like in the present case.

The easy way to remove it is to add the following to the Advanced tab in Moove settings,  Raw SCSS - 

footer#page-footer .copyright  {display:none;}

which I have done now. 

 

a) In the footer when you click on the mailbox it goes to MS edge account and not to the given mail id.

The footer link for contact us is a mailto: link. This will work properly if your browser is configured properly to handle mailto: links.

If you find that clicking that link takes you to your microsoft account, you might be using microsoft edge browser? In that case,

https://christianheilmann.com/2020/07/20/quick-tip-how-to-make-mailto-links-open-in-gmail-in-microsoft-edge/

 

b) Phone when clicked goes to a number which has this number added before the number given "662453:"

Just like the contact us email link is a mailto: link, the telephone link is a tel: link, which again needs to be configured correctly in your browser for it to work.

The incorrect behaviour is also because the tel: link is not configured properly. Only the number should be entered, and not text like "Mobile:".

https://www.elegantthemes.com/blog/wordpress/call-link-html-phone-number

 

c) Although I have a Web whatsapp installed on my laptop it says the link is wrong. On mobile it works.

One workaround would be to enter the whatsapp phone number as

00919876543210 - without spaces and without the + symbol, using 0091 instead of +91 - then I think it works.

 

d) When one clicks on Google Play icon for mobile app. the user cannot return to the front page but goes to the site home page. To return to the front page he has to login and logout.

I think this is related to what we have discussed earlier, Moodle shows a different page when one is logged in as compared to when logged out. Also, user is automatically logged in to guest login if any of the internal links which have guest access are clicked.

 

e) Is it possible to remove the App. Store link or the version 4 of moodle supports mobile app. on IOS devices.

Yes, we can remove if necessary using custom CSS. We probably need to link to the India site, like

https://apps.apple.com/in/app/moodle/id633359593

(the US site https://apps.apple.com/us/app/moodle/id633359593 does not seem to load for users in India)

7) I have added sample videos in some blocks to check. However if you open a video for viewing one cannot return to the front page but lands on the site home page. (Front page means non logged in page). One has to log in and log out to land again on the front page.

Please see how the marketing block looks like on the production site. I don't think there is any change in functionality 

https://oursite.org/admin/settings.php?section=themesettingmoove#theme_moove_frontpage

 Whenever I log into the site I get a mail saying new sign in. May be a version 4.0 addition. Can it be disabled or it will remain as default.

You can probably change this behaviour from

https://oursite.org/message/notificationpreferences.php

 

 

Linux ls - listing files sorted by date

Since some email notifications had not arrived, I did not have a ready reference to know which were the latest auto-encoded files. PB suggested looking in the server directory - and I could sort by date and go pagewise to get what I needed, 

ls -lt | more

problems with mysql server on Windows Server 2016 and workaround

The latest MySQL community editions - 8.0.22, 8.030 etc - which use installer version 1.6 - don't install properly on Windows Server 2016. They install fine on Windows 11. 

The error seen in the Log tab of the installer is,

Option --authentication-policy was set to an invalid value

(and I suspect when that is corrected, another error comes up.)

By trial and error, found a workaround, which was to install a version which was working fine on Windows Server 2016 - this post uses 8.0.19, so that was working. After the installation, after verifying that the server is running, we can allow the installer to update, and then after the installer updates itself, we can update the server to 8.0.30 without issues. The upgrade does some config file changes, I think. 
 

Some notes:

1. When reinstalling, we must delete both C:\Program Files\Mysql directory as well as C:\ProgramData\Mysql  before reinstalling if we want a fresh install with a fresh root password.
   
   
   
2. Found
   c:\programdata\mysql\mysql server 8.0\my.ini
   which had
   authentication_policy=0.0
   According to https://dev.mysql.com/doc/refman/8.0/en/multifactor-authentication.html the valid default should be
   '*,,'
   but in our installation, it is set to 0.0
   
   
Tried commenting it out, then tried running from command line as mentioned in the ini file, (but correcting the path),
mysqld --defaults-file="c:\programdata\mysql\mysql server 8.0\my.ini"
That  seemed to start, then immediately stop the server. So, no improvement. Probably there were other things wrong for Win Server 2016... 

Windows password - never expire

Windows passwords expire by default in 6 months. To prevent this, https://www.itechtics.com/set-windows-local-user-account-passwords-to-never-expire/

lusrmgr.msc --> Right-click relevant user --> Properties --> General tab --> (check) Password never expires

Windows brute-forcing on RDP ports

Noticed some events on one of our Windows servers -

4625(F) An account failed to log on. (Windows 10) - Windows security | Microsoft Docs

How solve Brute Force through IIS NTLM authentication scheme - Microsoft Q&A

Stopping the RDP service seems to stop it.