Official documentation at https://docs.moodle.org/311/en/Upgrading is a bit sparse on details like where to locate the plugins - I’m attempting to do this manually using
find . -name “customcert”
and so on for each of the “additional plugins” found on the plugins overview page at
https://oursite.tld/admin/plugins.php?updatesonly=0&contribonly=1This youtube video gives a good overview of how to manage a moodle installation and upgrades with git - https://www.youtube.com/watch?v=npks4TngSvo which gives a good presentation, tips for upgrading, a rapid summary and explanation for the documentation at https://docs.moodle.org/311/en/Git_for_Administrators
Creating the git directory and doing a test run with devel2.oursite.tld, which has the home dir /var/www/site - commands -
cd /var/www
sudo mkdir sitegit
sudo chown azureuser:www-data sitegit
git clone git://git.moodle.org/moodle.git sitegit
cd sitegit
git branch --track MOODLE_311_STABLE origin/MOODLE_311_STABLE
git branch --track MOODLE_39_STABLE origin/MOODLE_39_STABLE
git checkout MOODLE_39_STABLE
# first make sure 3.9 works, then upgrade 3.11List of additional plugins and commands to copy them -
cp -R ../site_learning/mod/customcert ./mod
# have to copy the moove pix_plugins after copying moove theme first
cp -R ../site_learning/mod/game ./mod
cp -R ../site_learning/mod/hvp ./mod
# have to copy the moove pix_plugins after copying moove theme first
# not migrating admin/tool/objectfs
cp -R ../site_learning/course/format/onetopic ./course/format
cp -R ../site_learning/report/coursesize ./report
cp -R ../site_learning/theme/moove ./theme
cp -R ../site_learning/theme/moove/pix_plugins/mod/customcert ./theme/moove/pix_plugins/mod
cp -R ../site_learning/theme/moove/pix_plugins/mod/hvp ./theme/moove/pix_plugins/mod
# not migrating local/azure_storage
# not migrating local/edwiserbridge
cp ../site/config.php .Change owner group from azureuser:azureuser to azureuser:www-data
sudo chown -R azureuser:www-data *Take a backup and create symlinks for easy switching between old and versions.
cd ..
sudo mv site site_old && sudo ln -s site_old site
# sudo chown azureuser:www-data site < - - - this doesn’t workCheck that the site is working, and that the version is seen as 3.9.2+ at https://devel2.oursite.tld/admin/index.php?cache=0
Site Administration -> notifications.Change over to the new code
sudo rm site && sudo ln -s sitegit siteCheck if Site Administration -> notifications reflects the new code - it prompts for update for minor upgrade. Have to go plugin by plugin and upgrade as needed. This took just a couple of minutes.
Now removed those plugins which we did not copy over, by going to Site Administration -> Plugins -> Plugins overview -> Additional plugins and uninstall for everything except those in step 4.
customcert
game
hvp
onetopic
coursesizeNow trying upgrading directly to 3.11 using git checkout -
git checkout MOODLE_311_STABLEOnly warning seen was PHP setting max_input_vars is recommended to be at least 5000. Clicked through all the prompts, Notifications page shows 3.11
Cron not run for 5 minutes error message went away after a couple of minutes. But some sort of css issue, page formatting jumbled. Purged caches with
sudo -u www-data php admin/cli/purge_caches.php
But still the issue persists. Looks like the accessibility toolbar.https://moodle.org/mod/forum/discuss.php?d=335856 gave the clue of ownership issues. Did Step 5. again. Stil not solved. https://moodle.org/mod/forum/discuss.php?d=325116 suggests deleting the theme. Deleted by
cd /var/www/sitegit/theme
ls
rm -R adaptable
rm -R moove
That solved the issue. Adaptable theme is not available for 3.11, perhaps that was the issue. Again installed moove theme manually, and changed to moove theme, OK.Trying to upgrade customcert,
Validating mod_customcert ... Error
[Warning] Target location already exists and will be removed [/var/www/sitegit/mod/customcert]
[Error] Write access check [/var/www/sitegit/mod/customcert] Installation aborted due to validation failurehttps://moodle.org/mod/forum/discuss.php?d=366460 pointed to permission issues. Found that mod/customcert did not have w set for group, so set for all mod with
cd /var/www/sitegit/mod
chmod -R g+w *
Solved.Found php.ini file location /etc/php/7.4/apache2/php.ini from https://devel2.oursite.tld/admin/phpinfo.php and modified to remove the warning about max_input_vars in step 12.
For migrating oursite.tld, additional steps of copying over data from azure blob storage to local disk. Rclone would help to copy in advance and later sync only those files which have been added later.
Compared speeds of downloads with oursite.tld without Cloudflare caching and devel2 with Cloudflare caching - not significantly different especially if cloudflare caching is on.
11 sec for file.pptx (4.8 MB) from devel2 (3.5 Mbps)
60 sec for afile.ppt (35.7 MB) from oursite (4.76 Mbps)
48 sec for afile.ppt (35.7 MB) from devel2 (5.95 Mbps)
So, migrating oursite to local disk from Azure blob storage will not have too much of a speed penalty. Is likely to save costs also since Azure storage charges for transfers and so on.In order to migrate oursite.tld filedir from Azure Blob storage to local HDD, started copying using a SAS token and SAS URL generated from the Azure portal.
portal.azure.com -> Resource groups -> site-rgname -> sitelearning (storage account) -> Containers -> sitelms -> Shared Access Tokens -> Generate SAS token and URL.On the server, installed rclone and set it up with
sudo apt install rclone
rclone config
setting up a new remote called azurecontainer by passing the SAS URL when prompted.Created directory structure by copying from /var/www/site_learning_files to /var/www/site_data_disk/site_learning_data
Tested rclone sync from the Azure container to the 00 subdirectory of filedir with
rclone sync azurecontainer:sitelms/00 filedir/00 -P
Worked, finished in less than a second.Started screen, and then ran
rclone sync azurecontainer:sitelms/ filedir/ -P
Showed ETA of 18 minutes for 34 GB. Total is 54 GB, so would probably take longer. Actual time taken was around 45 minutes for 54 GB.
Can sync again just before doing the migration in a few seconds. And then can remove the remote using
rclone config delete azurecontainerCreating git directory for site_learning just like step 3:
cd /var/www
sudo mkdir site_learning_git
sudo chown azureuser:www-data site_learning_git
git clone git://git.moodle.org/moodle.git site_learning_git
cd site_learning_git
git branch --track MOODLE_311_STABLE origin/MOODLE_311_STABLE
git branch --track MOODLE_39_STABLE origin/MOODLE_39_STABLE
git checkout MOODLE_39_STABLE
# first make sure 3.9 works, then upgrade 3.11List of additional plugins and commands to copy them, as in step 4, except the last line for config.php which should be from site_learning -
cp -R ../site_learning/mod/customcert ./mod
# have to copy the moove pix_plugins after copying moove theme first
cp -R ../site_learning/mod/game ./mod
cp -R ../site_learning/mod/hvp ./mod
# have to copy the moove pix_plugins after copying moove theme first
# not migrating admin/tool/objectfs
cp -R ../site_learning/course/format/onetopic ./course/format
cp -R ../site_learning/report/coursesize ./report
cp -R ../site_learning/theme/moove ./theme
cp -R ../site_learning/theme/moove/pix_plugins/mod/customcert ./theme/moove/pix_plugins/mod
cp -R ../site_learning/theme/moove/pix_plugins/mod/hvp ./theme/moove/pix_plugins/mod
# not migrating local/azure_storage
# not migrating local/edwiserbridge
cp ../site_learning/config.php . cp ../site_learning/*.html . The last line is to copy site policy agreements if any, if they are located in the root directory.Like step 5, change owner group from azureuser:azureuser to azureuser:www-data
sudo chown -R azureuser:www-data *Take a backup and create symlinks for easy switching between old and versions just like step 6.
cd ..
sudo mv site_learning site_learning_old && sudo ln -s site_learning_old site_learningSite working, but showed an initial error message on first login, later none. This concludes the pre-maintenance mode section.
Beginning the migration - Put the site in maintenance mode from Site Administration -> Server -> Maintenance mode.
Took a backup of the database by logging in to the database VM,
Commands from https://hnsws.blogspot.com/2020/12/mysql-and-postgresql-command-line-cheat.html
ssh site_admin@40.90.169.113
mysqldump -u site_db_admin -p site_learning_master_db -r LMSbk20210808.sql
(took just a few seconds, sql file was 333 MB in size.)Logged out of the Database VM, back on the php VM, equivalent commands to step 8 -
sudo rm site_learning && sudo ln -s site_learning_git site_learningNeed to sync the local data directory with the latest changes on Azure blob storage -
screen -x
rclone sync azurecontainer:sitelms/ filedir/ -P
Transferred: 149.056M / 149.056 MBytes, 100%, 22.964 MBytes/s, ETA 0s
Errors: 0
Checks: 15701 / 15701, 100%
Transferred: 148 / 148, 100%
Elapsed time: 6.4sMade sure the permissions were proper -
cd /var/www/site_data_disk
sudo chown -R azureuser:www-data site_learning_data
sudo chmod -R 775 site_learning_dataBefore starting the upgrade, we need to change config.php to point to the new data directory.
cd /var/www/site_learning_git
nano config.php
Changed the line
//$CFG->dataroot = '/var/www/site_learning_files';
To
$CFG->dataroot = '/var/www/site_data_disk/site_learning_data';
And commented out
//$CFG->alternative_file_system_class = '\tool_objectfs\azure_file_system';Copied the files from the existing data directory, other than the filedir directory, to the new data directory -
cd /var/www/site_data_disk/site_learning_data
cp -Rv /var/www/site_learning_files/lang .
cp -Rv /var/www/site_learning_files/models .
cp -Rv /var/www/site_learning_files/muc .Then started the upgrade process from the web console. Got
Fatal error: $CFG->dataroot is not specified in config.php! Exiting. https://moodle.org/mod/forum/discuss.php?d=370441
Indicates permission issues for the parent dir. Tried
sudo chown -R www-data:www-data /var/www/site_data_disk
sudo chown -R www-data:www-data /var/www/site_data_disk/site_learning_data
Still no. Then tried
sudo chown www-data:www-data config.php
And tried cloudflare removing the proxying for learning - still no go.Changed back the config file with
cd /var/www
sudo rm site_learning && sudo ln -s site_learning_old site_learning
Then the site works.Edited the file again to modify the existing line in /var/www/site_learning_git/config.php instead of creating a new line - that worked. Issue would probably be CRLF (line ending character) related, since the original config.php was probably created on Windows and I was editing it on Linux.
Did step 33 again -
sudo rm site_learning && sudo ln -s site_learning_git site_learningWorking. Then uninstalled azure_storage, objectfs and edwiserbridge from plugins using the web interface, from Site Administration -> Plugins -> Plugins Overview -> Additional plugins
Changed to Boost theme to prevent errors in moove theme during upgrade from Site Administration -> Appearance -> Theme selector
Then the upgrade to 3.11 by running
cd site_learning_git
git checkout MOODLE_311_STABLEThen doing the upgrade from the web interface by clicking on Site Administration and following the prompts. When trying to upgrade the plugins, got the Error that the directory is not writable. So, made sure all the git subdirectories were writable by the www-data group with
sudo chmod -R 775 /var/www/site_learning_gitRemoved maintenance mode.
Changed theme back to Moove.
Changed cloudflare back to proxied. Done.
Cleaning up -
Changed permissions of config.php to read-only
sudo chmod -w /var/www/site_learning_git/config.php
sudo chmod g-w /var/www/site_learning_git/config.phpTry to not have 777 permissions on the moodledata directory,
sudo chmod 755 /var/www/site_data_disk
sudo chmod 755 /var/www/site_data_disk/site_learning_data
(both are chown www-data:www-data)
Site seems to be working fine.
Sunday, August 08, 2021
detailed steps for moodle upgrade using git
Thursday, August 05, 2021
verbose display while booting with rEFInd
The Macbook running Linux Mint 19 was exhibiting some random issues, maybe due to running out of memory, and was needing hard restarts. I wanted to see the results of disk checking while booting, so tried looking up verbose booting options. For GRUB, the advice is to remove the "quiet splash" options from /etc/default/grub
In my case, since I use rEFInd, I edited /boot/refind_linux.conf and removed the quiet splash options. That prevented the logo from being displayed, but still the diagnostic boot text was not being displayed without pressing escape. Then removed the vt.handoff=7 option also because it would be going to the graphics display number 7. So, now it works, with quiet splash vt.handoff=7 removed from the standard options.
"Boot with standard options" "root=UUID=of-my-disk ro"
"Boot to single-user mode" "root=UUID=of-my-disk ro quiet splash vt.handoff=7 single"
"Boot with minimal options" "ro root=UUID=of-my-disk"
Wednesday, August 04, 2021
parse csv line by line
var entirecsvfilestring = csvfile.getBlob().getDataAsString();
Logger.log('Loaded the entire csv to memory.');
// https://stackoverflow.com/questions/21711768/split-string-in-javascript-and-detect-line-break
var numberOfLineBreaks = (entirecsvfilestring.match(/\n/g)||[]).length;
Logger.log('Number of line-breaks: ' + numberOfLineBreaks);
var splitlines = entirecsvfilestring.split("\n");
var data_range = [];
var data;
for (var linei = 1; linei < splitlines.length; linei++) {
// skipping the first line, linei=0
if (splitlines[linei].length < 25) {
continue;
// don't create the asset if the line is too short to contain relevant data.
// This can happen if there are blank lines at the end of the csv.
}
data = parseCsv(splitlines[linei], ',');
Logger.log('Parsed line %s', linei);
.... etc
// https://gist.github.com/simonjamain/7e23b898527655609e5ff012f412dd50 function parseCsv(csvString, delimiter) {
var sanitizedString = csvString.replace(/(["'])(?:(?=(\\?))\2[\s\S])*?\1/g, function(e){return e.replace(/\r?\n|\r/g, ' ') });
return Utilities.parseCsv(sanitizedString, delimiter)
}
Saturday, July 24, 2021
Google Apps Scripts and Schroedinger's cat
Apologies for the title - it's just a tongue-in-cheek reference to the wavefunction collapse referred to in the classic Quantum Mechanics thought example where observing the phenomenon causes it to behave differently - the reference is not really accurate for this bug in Google Docs API, it's just a joke!
The bug - A google doc was being created using a google apps script which copied a template and copied elements from other google docs, and was sometimes resulting in blank output documents.
The initial observed behaviour was truly bizarre - once the script completes, if we immediately open the GDoc file it has created, either directly by browsing to My Drive -> Recents and the file , or programmatically by calling DriveApp to make a copy, the GDoc becomes blank. On the other hand, if we first open another GDoc - it can be any other GDoc - allow it to load fully, close it and then open the newly created GDoc, then the doc is not blank, it has all the data.
Even after waiting for 5 minutes and then opening the GDoc doesn't help - it becomes blank if the procedure above is not followed.
If we open another GDoc, but close it before it loads fully, and then open the MLP GDoc, again it becomes blank.
After a lot of trial and error, found that this was due to two separate causes.
1. The template had not been shared with the user running the script as Editor - it was shared as read-only. This was causing the script to return a "Action not allowed" error at the very end of the script just as it exits. getChild(j) and similar functions apparently need Editor access for reliable operation.
2. If the destination directory is a shared folder, or if the destination folder is open in another window or tab, blank documents result.
The second issue is the one which made me think of Schroedinger's cat. If you watch the document as it is being created, the script behaves differently!
Workaround currently is to create it in a folder not shared with anyone (root folder) and ensure that folder is not open in any other window while the script is running. If the script is interrupted, even documents previously created "successfully" would turn out to be blank. So, we have to ensure the script doesn't time out after half an hour or stop unexpectedly due to any other reason.
Thursday, July 22, 2021
app with notification support - details about Moodle notifications
Copy-pasting from some correspondence about a Moodle site and notifications desired from it, by email and by mobile app notifications -
https://moodle.org/plugins/
Saturday, July 03, 2021
configure postfix to send mail using gmail or PHP alternative, send email for low disk space alert
Found this article describing all the steps needed to make postfix send emails via Google's SMTP servers,
https://www.linode.com/docs/guides/configure-postfix-to-send-mail-using-gmail-and-google-workspace-on-debian-or-ubuntu/
or
https://devanswers.co/configure-postfix-to-use-gmail-smtp-on-ubuntu-16-04-digitalocean-droplet/
But the implementation on our server used the simpler PHPMailer method.
https://phppot.com/php/send-email-in-php-using-gmail-smtp/
One of the use cases was to alert when disk space reaches critically low levels - https://stackoverflow.com/questions/60264983/email-notification-when-disk-drive-reaches-a-certain-percentage-laravel
Wednesday, June 30, 2021
caution with Moodle VMSS
The Azure VMSS (Virtual Machine Scale Set) Moodle deployment in combination with automated site backups has to be used with caution. The object storage filesystem instructs the administrator to create SAS keys with only read and write permissions, and it looks like this would cause the moodle scripts which try to do deletes to fail. So, in our case, when automatic site backups were enabled, blob storage usage began to balloon.
Also, occasionally, the backup would cause some issue due to which the object storage filesystem would stop writing to blob storage and would instead fill up the local hard disk, causing moodle to complain "session partition full". One quick check and the first place to delete would be the temp directory, especially if backup is turned on.
Even after running some scripts (a bash script is mentioned in this forum thread) and deleting all the unnecessary backups and orphaned files, blob capacity did not come down, only container size came down. So, my brute-force solution was to avoid blob storage and use local hard disk storage instead.
Tuesday, June 29, 2021
Chrome blocking http links
Google Chrome browser now blocks http links from pages served over https. Even if the http linked pages have https redirect enabled, Chrome continues to block. Only way out would be to replace all http links with https links.
HTTP redirect was with the rule which was created by certbot etc,
RewriteEngine on
RewriteCond %{SERVER_NAME} =our.server.name
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
Friday, June 25, 2021
backing up to Google drive with rclone
Backing up from a server to google drive using rclone directly on the server, using screen so that I don't have to be logged in for the entire multi-hour process - 1 GB takes around 2 hours. So it will take another couple of days for the process to be completed.
Commands were like
rclone copy srcDirectory gdrivearchivercloneremotename:"backupDir/destDirectory"
and so on.
Example output:
Transferred: 7.493G / 7.493 GBytes, 100%, 396.555 kBytes/s, ETA 0s
Transferred: 46441 / 46441, 100%
Elapsed time: 5h30m20.6s
Sunday, June 20, 2021
finding which files are causing disk to get full from linux command line
One of our servers ran out of disk space, so in order to diagnose the issue from the terminal, used the technique mentioned in this askubuntu post, that is,
du -cha --max-depth=1 / | grep -E "M|G"
Then, to reduce log file usage by journald, as this thread says,
edit /etc/systemd/journald.conf
Activate the SystemMaxUse= option there, e.g. as SystemMaxUse=100M to only use 100 MB.
service systemd-journald restart to activate the changed configuration.
Then, to clear up emails for root from /var/mail - they are mostly cron emails - various techniques discussed here, easiest would be to just empty it - but the safest way to do it might be using the mail command,
sudo su -
mail
delete *
q
which immediately cleared up all the root emails.
(By default, if you press enter and read a mail, it gets saved to an mbox file in that user's home directory, so in this case, /root/mbox )
Then, redirecting the cron job emails to /dev/null to prevent this from happening again, by appending
> /dev/null 2>&1
to all those cron jobs.
Thursday, June 17, 2021
adding SSL (https) to http mp3 streams - icecast-kh
The earlier reverse-proxy solution had hiccups when thousands of listeners connected. So, explored directly using HTTPS on icecast. Then PB discovered icecast-kh which has improved SSL certificate handling, without needing a restart when the certificate is updated -
an interesting fork of icecast2 called icecast-kh
https://github.com/karlheyes/icecast-kh
https://github.com/xiph/Icecast-Server/issues/20
https://github.com/AzuraCast/AzuraCast/issues/358 says the following:
8/5/2017
--autodetect SSL connections on incoming sockets. No need for in listen-socket now but is still there for compatability. (THIS IS BIG!!! We no longer need to use different ports for encrypted and unencrypted dramatically reducing complexity)
--add ssl-private-key in paths to allow for combined PEM or for separate SSL key/certificate files. (THIS IS BIG TOO! No longer need a separate process after updating let's encrypt to combine your fullchain and private cert into one pem file for icecast to read it!)
--select https/http URL in autogenerated m3u based on incoming request.
So, this was what was implemented, with a LetsEncrypt certificate. Compiled with
$ cd <directory where you have extracted the tar.gz>
$ ./configure --prefix=/home/ouruser/icecastkhSSL
$ make
$ make install
The SSL certificate is updated through change_ssl_certificate.sh
cat /etc/ssl/private/ourserver.tld.key /root/.acme.sh/ourserver.tld/fullchain.cer > /path/to/icecast.pem.letsencrypt
chown ouruser:ouruser /path/to/icecast.pem.letsencrypt
cp /path/to/icecast.pem /path/to/icecast.pem.backup
#echo "Backed up icecast.pem to icecast.pem.backup..."
cp /path/to/icecast.pem.letsencrypt /path/to/icecast.pem
Cron is run as root every day after acme script
43 0 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" > /dev/null
50 0 * * * /home/path/to/scripts/change_ssl_certificate.sh > /dev/null
Friday, June 11, 2021
server upgrade, certificates
We planned a streaming server upgrade. During the upgrade process, we would transfer the DNS to our web server, making it the temporary streaming server.
According to one of the replies at
the easiest way for temporary certificate transfer would be to just copy the certificates and icecast proxy server executable from our streaming server to our web server.
So,
1. copy the certificates and 2nd icecast server to the web server
2. check that the web server returns the correct certificate by locally setting /etc/hosts file DNS record of our stream url to the web server's IP address
3. on cloudflare, change the IP address from the streaming ip address to the web server's ip address for the stream url A record
After the upgrade is done, again do the steps 1, 2 and 3 for going from the web server to the streaming server. If everything goes well, the certificates and executable will remain on on the streaming server, and we will not need to copy them back.
Saturday, June 05, 2021
making GMail the default mail handler
Opening mailto links in GMail -
https://developers.google.com/web/updates/2012/02/Getting-Gmail-to-handle-all-mailto-links-with-registerProtocolHandler
(Not sure if this 2012 technique still works for Firefox and Chrome).
Firefox has a support page for this -
https://support.mozilla.org/en-US/kb/change-program-used-open-email-links
As also Chrome -
https://support.google.com/a/users/answer/9308783?hl=en
Friday, June 04, 2021
conditional formatting in Google sheets
https://stackoverflow.com/questions/21899516/how-to-highlight-cell-if-value-duplicate-in-same-column-for-google-spreadsheet
Monday, May 24, 2021
Opening a pdf as a Google Document using google apps script
This link has the steps to make Google apps script do OCR on the PDF -
But this generally gave pretty terrible results for me, and the formatting was completely lost.
For my use case, found that the PDF was being created from HTML, so direct conversion from HTML to GDoc gave good results -
assethtml += contentdata;
var ablob = Utilities.newBlob(assethtml, MimeType.HTML, "asset.html");
var AssetGDocId = Drive.Files.insert(
{ title: 'The name of the document',
mimeType: MimeType.GOOGLE_DOCS, parents: [{"id": destFolderID}] },
ablob ).id;
Wednesday, May 19, 2021
appending Google docs and adding header and footer
https://stackoverflow.com/questions/51252160/can-i-copy-the-contents-of-one-doc-to-a-specific-spot-in-another-doc-google-ap
Saturday, May 08, 2021
glitch.com for building web apps quickly
An interesting service to build Node.js web apps by "remixing" available code -
https://blog.glitch.com/post/google-docs-markdown-glitch
https://help.glitch.com/kb/article/17-what-are-the-technical-restrictions-for-glitch-projects/ - 1000 free project hours per month, 200 MB disk space, 4000 requests per hour.
https://flaviocopes.com/glitch/ has some interesting use cases - teaching, timed webhook and so on.
Friday, May 07, 2021
Getting started with Google Apps Script
A good example of using google apps scripts -
https://gist.github.com/colezlaw/29841885e7f2f49faa07
(also has link to explain google.script.run google script runner etc.)
Another excellent tutorial / collection of sample snippets is at
https://github.com/tanaikech/taking-advantage-of-Web-Apps-with-google-apps-script
There is also the google developers quick-start,
https://developers.google.com/docs/api/quickstart/apps-script
Delegating authority is mentioned at
https://developers.google.com/apps-script/guides/services/advanced#enable_advanced_services
Using service accounts to access api etc, step-by-step instructions are at
https://skaaptjop.medium.com/access-gsuite-apis-on-your-domain-using-a-service-account-e2a8dbda287c
and the gist is at
https://stackoverflow.com/questions/59435611/how-to-access-google-doc-api-with-service-account
Tuesday, May 04, 2021
calling a web app using curl in php
Just the syntax to use in php for using curl and making a GET or POST request to a google apps script - a slight modification from the answer given here, as curl_close was not needed for us -
$url = 'https://script.google.com/macros/s/ID_GOES_HERE/exec?optionname=OptionValue';
$schName = 'Some test';
$url = $url . urlencode($schName);
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl,CURLOPT_FOLLOWLOCATION, true);
$response = curl_exec($curl);
echo $response;
Monday, May 03, 2021
enabling desktop app notifications on linux
I had mistakenly clicked on "Don't show this message again" or something like that, and wanted to re-enable the notification shown when a VPN connection is made (or fails). The way to do it was mentioned at https://forums.linuxmint.com/viewtopic.php?t=89627
sudo gsettings set org.gnome.nm-applet disable-connected-notifications false sudo gsettings set org.gnome.nm-applet disable-disconnected-notifications false sudo gsettings set org.gnome.nm-applet disable-vpn-notifications false
file written to google drive by service account
When using a service account to create a new file and add it to Google drive, the file owner is the service account.
Can transfer ownership using Google Apps Script also.
https://stackoverflow.com/questions/65256980/google-drive-api-v3-php-client-transfer-file-ownership
restarting remote desktop via ssh or azure portal
Bs series burstable Azure VMs seem to have an endemic problem of the remote desktop service dying or refusing connections after the VM sees some heavy loads. Initially I had no idea why RDesktop was not connecting ... anyway, a couple of workarounds -
- install ssh server and leave the ssh service running
ssh into the machine when RDP has issues,
ssh adminuser@server.tld -p port
powershell
get-service termservice
get-service termservice -dependentservices
stop-service UmRdpService
restart-service termservice
start-service UmRdpService - install Windows Admin Center on the VM and connect via the Azure portal, restarting the same services as above. We need to use the "Connect with Public IP address" method on the Azure portal.
Sunday, May 02, 2021
limits for google apps scripts execution
A six-minute script execution time is mentioned at
https://developers.google.com/apps-script/guides/services/quotas (also many more limits)
but if we run the script from the Script Editor, the limit is 30 minutes. Probably since our Google Workspace is a non-profit one, so the "paid" limit applies. And a programmatic workaround (which I've not used till now.)
Saturday, May 01, 2021
handling duplicates in an sql query
There was a database in which we needed to join three tables, asset id -> subtheme id -> subtheme and the subtheme ids had duplicates as well as multiple subthemes mapped to an asset id. Getting a go-ahead from the content lead, the solution chosen was to just take a single subtheme, using select distinct - something like the first method at https://www.sisense.com/blog/4-ways-to-join-only-the-first-row-in-sql/
left join (select distinct on (asset_id) *
from asset_subtheme_xref
order by asset_id asc
) as asx on asx.asset_id = tma.asset_id
left join subtheme_master lsm on lsm.subtheme_id = asx.subtheme_id
Edit: Later on, multiple subthemes were desired. Then, instead of using select distinct in this way, we would need some sort of subquery similar to the one used below for another field, using concat to add all the different subthemes together with CHR(10) - linefeed - between them - all enclosed in parentheses -
(select string_agg(concat(btwo.name,',',stwo.name,',',sutwo.NAME,',',ctwo.name), CHR(10))
as "Asset tagged to Board Standard Chapter"
from asset_bssc_xref tabxr
left join chapters ctwo on ctwo.chapter_id = tabxr.chapter_id
left join board_master btwo on btwo.board_id = ctwo.board_id
left join standard_master stwo on stwo.standard_id = ctwo.standard_id
left join subject_master sutwo on sutwo.subject_id = ctwo.subject_id
where
tabxr.asset_id = tma.asset_id) as "Asset Tagged to"
installing ssh client and server on Windows Server 2016
https://hostadvice.com/how-to/how-to-install-an-openssh-server-client-on-a-windows-2016-server/
Or maybe needs this,
https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH
or other solutions listed at
or maybe the powershell method listed at
https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse