Sunday, August 29, 2021

cloudflare proxied self-signed web server issues

There was a problem with one of our websites set up with a different domain, where another site was being served up instead of the site configured with relevantdomain.conf virtualhost in apache. Turned out the issue was due to the way in which we had set up cloudflare proxying and SSL.

The apache server had been set up with a self-signed certificate, cloudflare was using flexible SSL mode, and a page rule had been applied in cloudflare for http to https redirection. But the web server had been configured with the self-signed cert for another domain. Maybe this was the reason for cloudflare to serve up the default.conf domain instead of the relevantdomain.conf.

Anyway, tidying up by using
certbot-auto --apache
and installing letsencrypt certificates for all the domains on that server fixed up the issue. The usual caution - when doing automatic http validation, turning cloudflare proxying off may be required. But for renewing the domains, this doesn't seem to be necessary, as tested with 
certbot-auto renew --dry-run

Later set up automatic renewal with 
45 2 * * 6 cd /etc/letsencrypt/ && ./certbot-auto renew && /etc/init.d/apache2 restart

Tuesday, August 17, 2021

not booting after timeshift restore

An upgrade from Linux Mint 19.3 to 20 did not end well. dpkg got stuck with some circular dependencies, probably because I had not removed all "foreign" packages before the upgrade. Then tried a fresh install of Linux Mint 20.1 and and upgrade to 20.2. Unfortunately, there seems to be a regression for Macbook support - the screen becomes corrupted after suspend and wake up. 

So, I tried to restore the timeshift backup I had made before the upgrade. That took a couple of hours, since I had backed up 40+ GB of home directory also. But then the system wouldn't boot into that restored partition.
Error message:  Alert: UUID=<whatever> does not exist, dropping to a shell. 



After looking at many places like here and here, finally I thought of checking the refind file boot/refind_linux.conf since I was booting with refind. That immediately showed up the issue. The boot/refind_linux.conf on the relevant partition, /dev/sda3 in this case, was pointing to a different UUID compared to the UUID reported by 
sudo blkid

So I just had to boot into another working Linux installation, edit this file boot/refind_linux.conf on /dev/sda3 with the correct UUIDs, and save it. The next boot worked fine. 

Friday, August 13, 2021

making a book readable on the Kindle from a set of articles

There are some pages with links to a large number of articles, like this one. Looking around for easily converting all those articles into a single book readable on an e-ink reader like the Kindle, my first trial was with Calibre and its News feature. Just adding that html page as the source did not work - the created ebook only had that page and nothing else.

Next, trying creating an RSS feed with fivefilters and giving that feed to Calibre - limited to 5 links, so not sure how it would work with the hundreds of old articles. 

Trying HTTrack to download pages - was taking a long time due to downloading CSS, images, etc. And we were not really interested in images.

So, used DownloadThemAll, choosing to download only *.htm and *.html linked from the index page, created a table of contents html page as noted in the FAQ. But Calibre complained of broken image links and stopped the conversion. 

Then tried converting all the html to txt using html2text - did not work, maybe my commandline piping was incorrect? Next, tried converting all the html to txt using HtmlAsText, running it as 
wine HtmlAsText.exe
This worked without a hitch. Then, made another table of contents html file with all the html files replaced by the txt files in the table of contents. But that conversion too ended up with only the table of contents being added to the ebook. 

Next, tried
cat *.txt > combinedfile.txt
and converted the combinedfile.txt. This created a combined ebook, but with ugly hard line-breaks every other line. Then, chose to enable heuristics processing with all options, and converted again. This time, got good results. With heuristics processing, the conversion takes longer, around 5-6 minutes for a 3 MB text file.  

So, to sum up - 
  • DownloadThemAll
  • HtmlAsText
  • cat
  • Calibre with heuristics enabled.


Thursday, August 12, 2021

assigning roles to users on Moodle

On one of our Moodle installations, there was a request to change permissions of a couple of users - to assign the "manager of course category" status of one user to another. Though Moodle has an easy way to make a user a manager of the entire site, Site Administration -> Users -> Assign system roles - this particular permission assignment was a bit more tricky to navigate. One solution turned out to be:

  1. Navigate to a course for which this user has manager status as an Admin or Manager
  2. Choose Course "Actions Menu" (Gear icon on right corner with the Moove theme)
  3. Choose "More" -> Users tab -> "Check permissions"
  4. Filter for the user whose permissions you want to change, click to select and click "Show this user's permissions"
  5. In this case, it showed, among other things, Manager in Category:TheRelevantCourseCategory
  6. Clicking on Category:TheRelevantCourseCategory, then again the "Actions Menu" (Gear icon on right corner with the Moove theme), Assign Roles
  7. That gives us a clickable table to assign roles (like Manager) in that particular category.

Sunday, August 08, 2021

detailed steps for moodle upgrade using git

  1. Official documentation at https://docs.moodle.org/311/en/Upgrading is a bit sparse on details like where to locate the plugins - I’m attempting to do this manually using
    find . -name “customcert”
    and so on for each of the “additional plugins” found on the plugins overview page at
    https://oursite.tld/admin/plugins.php?updatesonly=0&contribonly=1

  2. This youtube video gives a good overview of how to manage a moodle installation and upgrades with git - https://www.youtube.com/watch?v=npks4TngSvo which gives a good presentation, tips for upgrading, a rapid summary and explanation for the documentation at https://docs.moodle.org/311/en/Git_for_Administrators

  3. Creating the git directory and doing a test run with devel2.oursite.tld, which has the home dir /var/www/site  - commands -
    cd /var/www
    sudo mkdir sitegit
    sudo chown azureuser:www-data sitegit
    git clone git://git.moodle.org/moodle.git sitegit
    cd sitegit
    git branch --track MOODLE_311_STABLE origin/MOODLE_311_STABLE
    git branch --track MOODLE_39_STABLE origin/MOODLE_39_STABLE
    git checkout MOODLE_39_STABLE
    # first make sure 3.9 works, then upgrade 3.11

  4. List of additional plugins and commands to copy them -
    cp -R ../site_learning/mod/customcert ./mod
    # have to copy the moove pix_plugins after copying moove theme first
    cp -R ../site_learning/mod/game ./mod
    cp -R ../site_learning/mod/hvp ./mod
    # have to copy the moove pix_plugins after copying moove theme first
    # not migrating admin/tool/objectfs
    cp -R ../site_learning/course/format/onetopic ./course/format
    cp -R ../site_learning/report/coursesize ./report
    cp -R ../site_learning/theme/moove ./theme
    cp -R ../site_learning/theme/moove/pix_plugins/mod/customcert ./theme/moove/pix_plugins/mod
    cp -R ../site_learning/theme/moove/pix_plugins/mod/hvp ./theme/moove/pix_plugins/mod
    # not migrating local/azure_storage
    # not migrating local/edwiserbridge
    cp ../site/config.php .

  5. Change owner group from azureuser:azureuser to azureuser:www-data
    sudo chown -R azureuser:www-data *

  6. Take a backup and create symlinks for easy switching between old and versions.
    cd ..
    sudo mv site site_old && sudo ln -s site_old site
    # sudo chown azureuser:www-data site < - - - this doesn’t work

  7. Check that the site is working, and that the version is seen as 3.9.2+ at https://devel2.oursite.tld/admin/index.php?cache=0
    Site Administration -> notifications.

  8. Change over to the new code
    sudo rm site && sudo ln -s sitegit site

  9. Check if Site Administration -> notifications reflects the new code - it prompts for update for minor upgrade. Have to go plugin by plugin and upgrade as needed. This took just a couple of minutes.

  10. Now removed those plugins which we did not copy over, by going to Site Administration -> Plugins -> Plugins overview -> Additional plugins and uninstall for everything except those in step 4.
    customcert
    game
    hvp
    onetopic
    coursesize

  11. Now trying upgrading directly to 3.11 using git checkout -
    git checkout MOODLE_311_STABLE

  12. Only warning seen was PHP setting max_input_vars is recommended to be at least 5000. Clicked through all the prompts, Notifications page shows 3.11

  13. Cron not run for 5 minutes error message went away after a couple of minutes. But some sort of css issue, page formatting jumbled. Purged caches with
    sudo -u www-data php admin/cli/purge_caches.php
    But still the issue persists. Looks like the accessibility toolbar.

  14. https://moodle.org/mod/forum/discuss.php?d=335856 gave the clue of ownership issues. Did Step 5. again. Stil not solved. https://moodle.org/mod/forum/discuss.php?d=325116 suggests deleting the theme. Deleted by
    cd /var/www/sitegit/theme
    ls
    rm -R adaptable
    rm -R moove
    That solved the issue. Adaptable theme is not available for 3.11, perhaps that was the issue. Again installed moove theme manually, and changed to moove theme, OK.

  15. Trying to upgrade customcert,
    Validating mod_customcert ... Error
      [Warning] Target location already exists and will be removed [/var/www/sitegit/mod/customcert]  
    [Error] Write access check [/var/www/sitegit/mod/customcert] Installation aborted due to validation failure

  16. https://moodle.org/mod/forum/discuss.php?d=366460 pointed to permission issues. Found that mod/customcert did not have w set for group, so set for all mod with
    cd /var/www/sitegit/mod
    chmod  -R g+w *
    Solved.

  17. Found php.ini file location /etc/php/7.4/apache2/php.ini from https://devel2.oursite.tld/admin/phpinfo.php and modified to remove the warning about max_input_vars in step 12.

  18. For migrating oursite.tld, additional steps of copying over data from azure blob storage to local disk. Rclone would help to copy in advance and later sync only those files which have been added later.

  19. Compared speeds of downloads with oursite.tld without Cloudflare caching and devel2 with Cloudflare caching - not significantly different especially if cloudflare caching is on.
    11 sec for file.pptx (4.8 MB) from devel2 (3.5 Mbps)
    60 sec for afile.ppt (35.7 MB) from oursite (4.76 Mbps)
    48 sec for afile.ppt (35.7 MB) from devel2 (5.95 Mbps)
    So, migrating oursite to local disk from Azure blob storage will not have too much of a speed penalty. Is likely to save costs also since Azure storage charges for transfers and so on.

  20. In order to migrate oursite.tld filedir from Azure Blob storage to local HDD, started copying using a SAS token and SAS URL generated from the Azure portal.
    portal.azure.com -> Resource groups -> site-rgname -> sitelearning (storage account) -> Containers -> sitelms -> Shared Access Tokens -> Generate SAS token and URL.

  21. On the server, installed rclone and set it up with
    sudo apt  install rclone
    rclone config
    setting up a new remote called azurecontainer by passing the SAS URL when prompted.

  22. Created directory structure by copying from /var/www/site_learning_files to /var/www/site_data_disk/site_learning_data

  23. Tested rclone sync from the Azure container to the 00 subdirectory of filedir with
    rclone sync azurecontainer:sitelms/00 filedir/00 -P
    Worked, finished in less than a second.

  24. Started screen, and then ran
    rclone sync azurecontainer:sitelms/ filedir/ -P
    Showed ETA of 18 minutes for 34 GB. Total is 54 GB, so would probably take longer. Actual time taken was around 45 minutes for 54 GB.
     

  25. Can sync again just before doing the migration in a few seconds. And then can remove the remote using
    rclone config delete azurecontainer

  26. Creating git directory for site_learning just like step 3:
    cd /var/www
    sudo mkdir site_learning_git
    sudo chown azureuser:www-data site_learning_git
    git clone git://git.moodle.org/moodle.git site_learning_git
    cd site_learning_git
    git branch --track MOODLE_311_STABLE origin/MOODLE_311_STABLE
    git branch --track MOODLE_39_STABLE origin/MOODLE_39_STABLE
    git checkout MOODLE_39_STABLE
    # first make sure 3.9 works, then upgrade 3.11

  27. List of additional plugins and commands to copy them, as in step 4, except the last line for config.php which should be from site_learning -
    cp -R ../site_learning/mod/customcert ./mod
    # have to copy the moove pix_plugins after copying moove theme first
    cp -R ../site_learning/mod/game ./mod
    cp -R ../site_learning/mod/hvp ./mod
    # have to copy the moove pix_plugins after copying moove theme first
    # not migrating admin/tool/objectfs
    cp -R ../site_learning/course/format/onetopic ./course/format
    cp -R ../site_learning/report/coursesize ./report
    cp -R ../site_learning/theme/moove ./theme
    cp -R ../site_learning/theme/moove/pix_plugins/mod/customcert ./theme/moove/pix_plugins/mod
    cp -R ../site_learning/theme/moove/pix_plugins/mod/hvp ./theme/moove/pix_plugins/mod
    # not migrating local/azure_storage
    # not migrating local/edwiserbridge
    cp ../site_learning/config.php . cp ../site_learning/*.html . The last line is to copy site policy agreements if any, if they are located in the root directory.

  28. Like step 5, change owner group from azureuser:azureuser to azureuser:www-data
    sudo chown -R azureuser:www-data *

  29. Take a backup and create symlinks for easy switching between old and versions just like step 6.
    cd ..
    sudo mv site_learning site_learning_old && sudo ln -s site_learning_old site_learning

  30. Site working, but showed an initial error message on first login, later none. This concludes the pre-maintenance mode section.


  31. Beginning the migration - Put the site in maintenance mode from Site Administration -> Server -> Maintenance mode.

  32. Took a backup of the database by logging in to the database VM,
    Commands from https://hnsws.blogspot.com/2020/12/mysql-and-postgresql-command-line-cheat.html
    ssh site_admin@40.90.169.113
    mysqldump -u site_db_admin -p site_learning_master_db -r LMSbk20210808.sql
    (took just a few seconds, sql file was 333 MB in size.)

  33. Logged out of the Database VM, back on the php VM, equivalent commands to step 8 -
    sudo rm site_learning && sudo ln -s site_learning_git site_learning

  34. Need to sync the local data directory with the latest changes on Azure blob storage -
    screen -x
    rclone sync azurecontainer:sitelms/ filedir/ -P
    Transferred:      149.056M / 149.056 MBytes, 100%, 22.964 MBytes/s, ETA 0s
    Errors:                 0
    Checks:             15701 / 15701, 100%
    Transferred:          148 / 148, 100%
    Elapsed time:        6.4s

  35. Made sure the permissions were proper -
    cd /var/www/site_data_disk
    sudo chown -R azureuser:www-data site_learning_data
    sudo chmod -R 775 site_learning_data

  36. Before starting the upgrade, we need to change config.php to point to the new data directory.
    cd /var/www/site_learning_git
    nano config.php
    Changed the line
    //$CFG->dataroot  = '/var/www/site_learning_files';
    To
    $CFG->dataroot  = '/var/www/site_data_disk/site_learning_data';
    And commented out
    //$CFG->alternative_file_system_class = '\tool_objectfs\azure_file_system';

  37. Copied the files from the existing data directory, other than the filedir directory, to the new data directory -
    cd /var/www/site_data_disk/site_learning_data
    cp -Rv /var/www/site_learning_files/lang .
    cp -Rv /var/www/site_learning_files/models .
    cp -Rv /var/www/site_learning_files/muc .

  38. Then started the upgrade process from the web console. Got
    Fatal error: $CFG->dataroot is not specified in config.php! Exiting. https://moodle.org/mod/forum/discuss.php?d=370441
    Indicates permission issues for the parent dir. Tried
    sudo chown -R www-data:www-data /var/www/site_data_disk
    sudo chown -R www-data:www-data /var/www/site_data_disk/site_learning_data
    Still no. Then tried
    sudo chown www-data:www-data config.php
    And tried cloudflare removing the proxying for learning - still no go.

  39. Changed back the config file with
    cd /var/www
    sudo rm site_learning && sudo ln -s site_learning_old site_learning
    Then the site works.

  40. Edited the file again to modify the existing line in /var/www/site_learning_git/config.php instead of creating a new line - that worked. Issue would probably be CRLF (line ending character) related, since the original config.php was probably created on Windows and I was editing it on Linux.

  41. Did step 33 again -
    sudo rm site_learning && sudo ln -s site_learning_git site_learning

  42. Working. Then uninstalled azure_storage, objectfs and edwiserbridge from plugins using the web interface, from Site Administration -> Plugins -> Plugins Overview -> Additional plugins

  43. Changed to Boost theme to prevent errors in moove theme during upgrade from Site Administration -> Appearance -> Theme selector

  44. Then the upgrade to 3.11 by running
    cd site_learning_git
    git checkout MOODLE_311_STABLE

  45. Then doing the upgrade from the web interface by clicking on Site Administration and following the prompts. When trying to upgrade the plugins, got the Error that the directory is not writable. So, made sure all the git subdirectories were writable by the www-data group with
    sudo chmod -R 775 /var/www/site_learning_git

  46. Removed maintenance mode.

  47. Changed theme back to Moove.

  48. Changed cloudflare back to proxied. Done.

  49. Cleaning up -
    Changed permissions of config.php to read-only
    sudo chmod -w /var/www/site_learning_git/config.php
    sudo chmod g-w /var/www/site_learning_git/config.php

  50. Try to not have 777 permissions on the moodledata directory,
    sudo chmod 755 /var/www/site_data_disk
    sudo chmod 755 /var/www/site_data_disk/site_learning_data
    (both are chown www-data:www-data)
    Site seems to be working fine.