showing memory usage by process and user

Using ps on the linux terminal, we can check out memory usage - 
https://www.networkworld.com/article/3516319/showing-memory-usage-in-linux-by-process-and-user.html

"sort is being used with the -r (reverse), the -n (numeric) and the -k (key) options which are telling the command to sort the output in reverse numeric order based on the fourth column (memory usage) in the output from ps. If we first display the heading for the ps output, this is a little easier to see."

ps aux | head -1; ps aux | sort -rnk 4 | head -5

resubscribing to letsencrypt emails

According to this thread,
https://community.letsencrypt.org/t/accidentally-unsubscribed/14682/14
the way to resubscribe after accidentally unsubscribing would be to change the registration to email+1@mydomain.com with

certbot update_account --email yourname+1@example.com

azcopy with sas tokens and more

I struggled a bit with using azcopy to copy from one Azure storage container to another. The issue finally turned out to be that when a new SAS token was created, the default permissions were not sufficient for what I wanted to do - needed to set the expiry time and read / write permissions correctly. After doing that, the command was:

azcopy cp "https://ourstorage.blob.core.windows.net/ourname/*" "https://ourdestination.blob.core.windows.net/destname?sp=racw&st=-snip-&se=-snip-&spr=https&sv=-snip-&sr=c&sig=-snip-%3D" --recursive

51.3 %, 4919 Done, 0 Failed, 5647 Pending, 0 Skipped, 10566 Total, 2-sec Throughput (Mb/s): 3685.9743

 

- finished in around 5 min.

updating an app on Google's Android Play App store

Updating an already existing app with an already existing developer account - the process was fairly intuitive. Older tutorials point to https://market.android.com/publish/Home - that url now redirects to https://play.google.com/console. 

Basically we need to click on the relevant app, choose View Releases Overview - Release Dashboard - Create new Release.

To edit the play store listing - the text shown on the play store - we need to scroll all the way down on the left-hand side, Grow - Store presence - Main store listing.

For creating the release, we need to sign with the same keystore and signature used to sign the earlier version, etc. as in this post.

comparing Azure pricing in different locations

An interesting site - https://azureprice.net/ Makes it easy to find the region with lowest pricing for a particular type of VM. 

As the site says, pricing is also different in different currencies. 

customizing and building the moodle app

After learning how to use an older node version, the moodle app could finally be customized and built - a follow-up to my failed trials here. 

The customization steps and the relevant modified files are in this location, with the changes being mentioned in the readme as follows:

1. Changes as per the moodle doc on compiling with AOT
Edit Oct 2022 - the link content seems to have changed, so here is the archived version.

2. Change to the build.gradle to force minSdkversion 22

ext.cdvMinSdkVersion = 22

to build with android studio. This can also be done in config.xml

3. Changing all ~ and ^ package dependencies in package.json (except cordova-android itself) to the exact package. Done by replacing

"~ with "

and

"^ with "

in package.json

Or else, all sorts of dependency issues while building the 2019 release in 2021. (The nvm use 11 may also suffice for solving the dependency issues.)

The build directory takes up more than 1 GB, lots of dependencies are downloaded. 

4. Customization - Changes as per the Configuration heading at this post,

config.xml

reduced SplashScreenDelay

and SplashShowOnlyFirstTime true

also, very important,

content src="https://sssvidyavahini.org" would make the app like a simple webview.

We don't want that, so leave the content src as it is, 

but change the siteurl in src/config.json

and onlyallowlistedsites true.

We also set android-minSdkVersion" value="22" in config.xml

google-services.json - changed the package name

and changed the version in src/config.json , ensured it is different, but same number of digits.

5. Changing logo and splash screen as per above url and also resources/android/icon-background.png and icon-foreground.png

and in src dir assets/img

Build steps - 

Set up environment variables as per

https://cordova.apache.org/docs/en/10.x/guide/platforms/android/

JDK tar.gz needed oracle login, created as my official email

Set JAVA_HOME

https://docs.oracle.com/cd/E19182-01/821-0917/inst_jdk_javahome_t/index.html

in .bashrc, added the following.

export JAVA_HOME=/home/mac/Downloads/jdk1.8.0_281
export PATH=$JAVA_HOME/bin:$PATH
export ANDROID_SDK_ROOT=/home/mac/Android/Sdk
export PATH=$ANDROID_SDK_ROOT/tools:$ANDROID_SDK_ROOT/tools/bin:$ANDROID_SDK_ROOT/platform-tools:$PATH

As mentioned in the cordova documentation, we can open a cordova project inside android studio for the final build - choose the import gradle project option, and choose the platforms/android directory.

But we need to edit www folder outside android studio, then copy over changes by doing cordova build - in our case, npx etc as below.

The build process is detailed in the following steps to build document:

# https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2

nvm install node 11.15.0

# https://www.sitepoint.com/quick-tip-multiple-versions-node-nvm/

nvm use 11

# this is very important - if starting again in another terminal window, must do nvm use 11

# or else all sorts of hard to diagnose errors occur during the build.

sudo apt-get install libsecret-1-dev

npm install 

# cordova.plugins.diagnostic: Diagnostic plugin - ERROR: ENOENT: no such file or directory, open '/home/mac/StudioProjects/LMSappBuildTrial/config.xml'

# added 1829 packages from 1022 contributors and audited 1958 packages in 98.592s

npx cordova prepare 

# Current working directory is not a Cordova-based project.

# https://stackoverflow.com/questions/21276294/cordova-current-working-directory-is-not-a-cordova-based-project

#mkdir www

npm install cordova

# this is also important. Without this, the www directory is not populated for the final prod build.

npm i -g cordova-res

# This is also important - without this, the image resources won't get processed

# config.xml was missing, so copied over with git

# No platforms added to this project. Please use `cordova platform add <platform>`.

npx ionic cordova platform add android --verbose

#Source path does not exist: resources/android/icon/drawable-hdpi-smallicon.png

#Error: Source path does not exist: resources/android/icon/drawable-hdpi-smallicon.png

# editing gitignore file to add the resources

npx cordova prepare

# ignoring the warning about conflict

npx gulp

npm start

# This is an optional step to check in a browser and verify.

# waiting till transpile started etc. then Ctrl+C

# If we wait till the end - nearly an hour on a 4 GB system? it opens in the default browser.

# https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2#Compiling_using_AOT

cp -v "changes made to moodleapp files/inside node_modules dir/@angular/platform-browser-dynamic/esm5/platform-browser-dynamic.js" "node_modules/@angular/platform-browser-dynamic/esm5"

cp -v "changes made to moodleapp files/inside node_modules dir/@ionic/app-scripts/dist/util/config.js" "node_modules/@ionic/app-scripts/dist/util/config.js"

# edited gitignore to not ignore this config.js, did git add -f.

npm run ionic:build -- --prod

# this takes a lot of RAM and a lot of time. 

# if possible, avoid swapping by closing all apps and clearning memory before doing this.

# REMEMBER to use nvm use 11 if doing this in a fresh terminal

#PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND 

# 3682 mac       20   0 3142252 2.215g  28948 R 131.5 59.6   6:34.48 node

#   build prod finished in 3128.04 s - this was on a 4GB machine

# on a 16 GB GCP instance, it took up around 1/3rd of memory - 5 GB+ - 

# and finished in just 5 or 6 minutes instead of 50 minutes.

# npx cordova run android

# Could not find an installed version of Gradle either in Android Studio,

#or on your system to install the gradle wrapper. Please include gradle 

#in your path, or install Android Studio

# So, I just imported the platforms/android directory using the Import Gradle project option in Android Studio 4.1.3, and built from there.

/dev/kvm permission denied fix for Android AVD unable to open in Android studio

I did not go through this, since my machine had only 4 GB RAM and was probably too RAM constrained to run the virtual device, but

https://16shuklarahul.medium.com/how-to-fix-kvm-permission-denied-error-on-ubuntu-18-04-16-04-14-04-f04a6e23c0cd

- install qemu-kvm and then give appropriate permission to the current user.

$ sudo apt install qemu-kvm

% add your user to the kvm group.

$ sudo adduser <username> kvm

% And then 

$ sudo chown <username> /dev/kvm

uninstall apk before installing with different signature

While developing android apps, we find that sometimes the apks we build don't get installed - the package manager just says Not Installed. The reason is apparently that if the same app is currently installed with a different signature, it will not get overwritten. 

Uninstalling via long press on the home screen on my LG Q6 was not sufficient.

To uninstall cleanly, the way was as listed at this page:

Settings - Apps - Select the app to uninstall - Choose Force Stop
Then choose Storage - Clear Cache - Clear Data
Return to the app screen - choose Uninstall

After doing this, the apk will install. If it doesn't there may be something wrong with the signing etc. which will be the subject of another post of mine. 

signing release build apk files in android studio

The procedure to sign apk files - if you just click on Generate Signed Bundle / APK, the wizard prompts you to create a keystore (or use an existing one) and create the signed build. We have to save that keystore safely, since we will need it for signing any updates to the app on Google's Android Play store (app store). 

But that is not enough - the apk would not install. We've also got to set the signing configuration in the Project Structure, check both v1 and v2 signature versions, and also look for any of the issues which are listed at this page. Mainly, as in this link - 
Project Structure - Modules - Signing tab - add a configuration, and under Build types tab, release build, under signing config, choose the one which we have added. 

loading software on a few hundred tablets

There was some discussion about loading software on some Android devices which were to be shipped to schools. 

Initial thought was to create ids like
OurPrefix.TAB.2021.01@gmail.com
to log on to the play store for the tablet which has asset id OurPrefix-TAB-01 etc.

We could then print out the password and put it in the box for the teachers to change if necessary.

This is required, since this is not a factory install, and factory install process is very different from manual install :) For a manual install, we have to log on to the play store with a particular account, etc etc. 

If we are to not create such ids, installing and then logging out of the account used to install can work for our free apps, but with the drawback that the apps will not update. For updates, the users would have to uninstall and reinstall. 

The next idea was to install all these apps using pre-downloaded APK files, so that the internet connection would not be a bottleneck in the prep process which was planned to be done on a single day for several hundred devices - taking the help of a score or more of volunteers. This too had the drawback of the apps not updating automatically from the Play store. 

Finally the idea of links to the Play store was mooted. 

• Connect via USB cable to the computer, Choose the "File transfer' USB usage preference, copy the tabletslinks.txt file to the Download directory on the tablet. 
• Navigate to the txt file using Google Files on the tablet, click on it and choose to open with Chrome. 
• Copy each line (each link), open a new tab in Chrome, paste in Chrome, hit Enter (while offline)
• Click the three vertical dots on the right hand side of the Chrome address bar, choose "Add to Home Screen"
• Write an appropriate short link title like Pdf for the adobe pdf reader link, Office for the office link, Teams/Meet/Zoom for the respective links.
• Drag and drop the link to the home screen.

The whole process was timed at less than 5 minutes from the time the tablet was booted up. And this was the final solution chosen.

web push api and running our own push notification server for Moodle - link dump

https://developers.google.com/web/ilt/pwa/introduction-to-push-notifications#push_api

https://docs.moodle.org/310/en/Messaging_settings#Mobile_notifications

https://github.com/airnotifier/airnotifier/wiki/Installation-3.x

change default app in Android

https://www.techadvisory.org/2014/01/how-to-set-a-default-app-on-android/

Open Settings - Apps - the relevant app

Scroll down and choose "clear defaults" 

Android studio Git checkout

For checking out at a specific commit - 
https://stackoverflow.com/questions/36517118/how-do-i-checkout-an-old-git-commit-in-android-studio

right-click relevant commit on the Git tab's history, choose 'Checkout revision <hash>'

keeping Windows and Linux dual-boot times correct

This 2011 post has the Kubuntu 10 way, and for current machines running systemd, the way to do it is, if choosing to make Linux use the hardware clock as local time, 
timedatectl set-local-rtc 1

fixing Filezilla error about LC_CTYPE environment variable

As described on this page, the solution was

sudo nano /usr/share/applications/filezilla.desktop

Find  the line
Exec=filezilla
and replace it with
Exec=env LC_ALL=en_US.UTF-8 filezilla

how to identify if a DLL is a debug or release build

Needed to check if one of our servers was gobbling up RAM due to a debug DLL, so checked and found this SO conversation about identifying debug / release builds DLLs. One of the replies mentions '.NET Assembly Information' by Rotem Bloom - https://github.com/jozefizso/AssemblyInformation and a more recent fork at https://github.com/tebjan/AssemblyInformation

Also a quick and dirty solution, "One way that could work for most people is to simply open the DLL/EXE file with Notepad, and look for a path, for example search for "C:\" and you might find a path such as "C:\Source\myapp\obj\x64\Release\myapp.pdb", the "Release" shows that the build was done with Release configuration."

repairing boot after installing windows

For Ubuntu and Ubuntu-based distros like Linux Mint, running the graphical boot-repair tool from a live CD / USB is a good option. In my case, I had to choose to boot with advanced startup and restart in Windows 10, or else the BIOS screen was not visible for me to boot from USB.

creating Windows installer USB drive from ISO - on Linux with woeusb

The usual route using unetbootin would need the USB drive to be partitioned as FAT32 (using gparted, the Disks utility on Linux Mint doesn't do it - and have to mount with Disks utility after gparted does the reformatting). But then the Windows 10 installation ISO contains a file installer.wim which is 5 GB, so it can't be written to the FAT32 volume. So we have to use ntfs - 
https://askubuntu.com/questions/162174/how-do-i-use-unetbootin-to-make-a-bootable-windows-usb-installer
or use woeusb.

(Detailed steps:

woeusb ppa gives error on install,
The following packages have unmet dependencies:
 woeusb : Depends: libwxgtk3.0-0v5 (>= 3.0.4+dfsg) but it is not installable
E: Unable to correct problems, you have held broken packages.

woeusb 
needs https://wimlib.net/

sudo apt install build-essential
./configure
gave
No package 'libxml-2.0' found

sudo apt install libxml2-dev

Cannot find libntfs-3g

sudo apt install ntfs-3g-dev

error: Cannot find libfuse

sudo apt install libfuse-dev

sudo ./woeusb-5.1.0.bash --device /home/path/Win10_20H2_v2_EnglishInternational_x64.iso /dev/sdb

wimlib-imagex: error while loading shared libraries: libwim.so.15: cannot open shared object file: No such file or directory

Solution was to run ldconfig, as found in
https://wimlib.net/forums/viewtopic.php?t=240

sudo ldconfig -v

)

Visual Studio publish and deploy ASP .NET Core app to IIS

This link has the basics on how to do the setup, and also some do's and don'ts for ASP .NET apps on IIS - https://docs.microsoft.com/en-us/aspnet/core/tutorials/publish-to-iis?view=aspnetcore-3.0&tabs=visual-studio

run explorer as Administrator

On Windows server, there is no "Run As Administrator" directly available for File Explorer. So, we can go to C:\Windows with File Explorer, and then right-click on explorer.exe and Run As Administrator. 

sql query to replace a substring

We needed to replace some Windows paths in a MySQL database. Following 
https://www.mysqltutorial.org/mysql-string-replace-function.aspx/

update tablename set tablename.MediaSource = REPLACE(tablename.MediaSource,
        m4a,
        mp3)
WHERE
    tablename.MediaSource like '%Dashboard%1%1%2%1%2.m4a';

and if we wanted to be more specific, the '\' in the Windows paths need to be escaped as four backslashes - following https://stackoverflow.com/questions/14926386/how-to-search-for-slash-in-mysql-and-why-escaping-not-required-for-wher - 

select * from tablename where tablename.MediaSource like '%Dashboard%1%1%2\\\\1\\\\2.m4a';

And the syntax for a count statement was like

select count(*) from (select * from tablename where tablename.MediaSource like '%Dashboard%m4a') myqueryname;

finding mysql data size

According to 

https://support.microfocus.com/kb/doc.php?id=7019203#

we can find the data directory from my.ini which would be in server directory in program files - the datadir line in my.ini

But on our install, it was in c:\programdata\mysql

and data was also there.

Notes on locally mounting Windows NTFS drive on dual boot Linux

In order to make the partition visible as a separate entry on the side bar, move mount point to /media/windows1 or something like that - 
https://forums.linuxmint.com/viewtopic.php?t=300597

https://www.google.com/search?q=umask+needed+to+make+vfat+drive+writable+by+all+users+on+linux

umask=0 for writable by all.

visual studio offline installer

Wanted a way to create an offline Visual Studio installer - found it is possible, but have to run Windows to go through the process at this link.

Adding google search to Firefox on Linux Mint

Linux Mint has removed Google from the search engines available by default, and we have to follow a "Microsoft-esque" convoluted procedure to make Google search available from the address bar or the search box - following https://www.techbrown.com/add-google-default-search-engine-firefox-linux-mint/

• Go to https://linuxmint.com/searchengines.php
• Scroll down to the bottom of the page and click on the Google logo
• The page which loads up then has instructions to right-click on the address bar and add Google search from the context menu. 
• Then we can choose to make Google the default search from the search box from the "Change search settings" gear icon under the search box also. 

Restoring Linux after Windows install

Following the instructions from https://forums.linuxmint.com/viewtopic.php?t=92409

"Boot from a LIVE DVD (in my case Linux Mint booting from USB key drive)

If you don't know the location of the linux partition, you can determine the partition that Mint is installed on by:

Open terminal and run

sudo gparted

Look for the large EXT4 partition and make a note of what it says on the far left such as sda6

Close gparted

In terminal enter:

sudo mount /dev/sda6 /mnt (replace sda6 with whatever is appropriate for your system)

sudo grub-install --root-directory=/mnt /dev/sda (note there is a space between mnt and /dev and do not put a number after sda)

Restart the computer and you should have your grub menu available again."

But in order to boot from a different medium when Windows 10 is installed, we may need to go through some hoops, as mentioned in this post.

customizing Moodle theme per course

Replying to a request of adding blocks to some courses on one of our Moodle servers - 

https://moodle.org/mod/forum/discuss.php?d=256667

Site administration>Appearance>Themes>Theme settings - can set it to allow course themes. 

Then I went to an example test course I created,

• Turned editing on,
• clicked on the gear icon,
• clicked on Edit settings,
• Scrolled down to Appearance, opened up the Appearance section,
• and in the appearance section, chose
• Force Theme : Boost.
• After doing that, I'm able to add blocks as per this video,
  https://www.youtube.com/watch?v=uByp1qqcWt8

But please note, not all types of blocks are supported on mobile - please see the list of supported blocks below.

https://docs.moodle.org/310/en/Moodle_App_Block_support

In this way, you can set your own look and feel for only some courses as per the method above, without disturbing the look and feel of the rest of the site.

finding and listing files without extensions in Linux

This post - 
https://askubuntu.com/questions/337964/list-all-files-that-do-not-have-extensions
uses ls to list all files without extensions.

Then there is the technique using find,
https://unix.stackexchange.com/questions/47151/how-do-i-list-every-file-in-a-directory-except-those-with-specified-extensions
to list those files other than those with specific extensions, which I used like

find nameofdirectory ! '(' -name '*.mp3' -o -name '*.docx' -o -name '*.zip' ')'

find nameofdirectory ! '(' -name '*.*'  ')'
lists directories also.

find nameofdirectory -type f ! '(' -name '*.*' ')' | wc -l 
for counting the total.

file transfer using remote desktop - RDP

Transferring a single zip file is much much faster than transferring a large number of small files. Another issue with transferring a large number of files this way is that some extra files are created - "which indicate that the file originated from the network". Zone Identifier files as explained in this post - https://apple.stackexchange.com/questions/378438/what-are-these-extra-zone-identifier-files-created-during-windows-remote-deskto

To enable file transfer from the Remmina RDP client, check the shared folder and enable a folder to share, in the Basic tab, and Sound Local in Advanced tab

Macbook not charging

Resetting the SMC - System Management Controller - is supposed to cure many issues related to battery etc. In my case it did not help - maybe because the Applecare person has removed the bulged battery?

https://support.apple.com/en-gb/HT201295

"with non-removable battery" - 

Hold down Shift + Ctrl + Opt , then hold down power button also, for 10 sec.

Release all, then press power button to turn on.

creating webview android apps, and angular to apk

Some links, tutorials etc for easily creating webview-based Android apps - 

• https://www.tutorialspoint.com/how-to-create-a-webview-in-android-app
• https://developer.chrome.com/docs/multidevice/webview/gettingstarted/ - the official doc
• https://developer.android.com/codelabs/basic-android-kotlin-training-change-app-icon#0
• https://developer.android.com/guide/webapps/webview.html#kotlin - Java and Kotlin code
• https://github.com/mgks/Kotlin-SmartWebView - used this to create a demo app (but doesn't have microphone support)

And for converting angular to apk, https://medium.com/@christof.thalmann/convert-angular-project-to-android-apk-in-10-steps-c49e2fddd29

Edit: 25 Dec 2021 - For another version of webview with upload support, 
https://github.com/delight-im/Android-AdvancedWebView

changing the name of an Android app

1. From https://stackoverflow.com/questions/5443304/how-to-change-an-android-apps-name

By changing the android:label field in your application node in AndroidManifest.xml - Please make sure that you change label:

android:label="@string/title_activity_splash_screen"

in your Splash Screen activity in your strings.xml file. It can be found in Res -> Values -> strings.xml

2. change the package name - from

https://stackoverflow.com/questions/16804093/rename-package-in-android-studio

• In your Project pane, click on the little gear icon
• Uncheck the Compact Empty Middle Packages option
• Your package directory will now be broken up into individual directories
• Individually select each directory you want to rename, and:
• Right-click it, Select Refactor, Click on Rename
• In the pop-up dialog, click on Rename Package instead of Rename Directory
• Enter the new name and hit Refactor
• Click Do Refactor in the bottom
• Allow a minute to let Android Studio update all changes
• Note: When renaming com in Android Studio, it might give a warning. In such case, select Rename All

3. SmartWebView specific method - https://github.com/mgks/Kotlin-SmartWebView/issues/3

To change the build package name, change

File > Project Structure > Modules > Default config > Application ID

A commit with the name change is here.

ASP .NET memory usage

An ASP .NET web app under development was using a lot of memory. Googling, found this resource from Microsoft to check - https://docs.microsoft.com/en-us/troubleshoot/aspnet/high-memory-level

send a post request with curl

https://github.com/thephpleague/oauth2-server/tree/master/examples

https://gist.github.com/subfuzion/08c5d85437d5d4f00e58

Since the default is application/x-www-form-urlencoded, the shortest would be
curl -d "param1=value1&param2=value2" -X POST http://localhost:3000/data

or with a data file,
curl -d "@data.txt" -X POST http://localhost:3000/data

some post install changes needed for moodle

I had made a test moodle server by copying over the database, moodledata and /var/www/html/moodleinstalldirectory to a different server. Additionally, the following changes needed to be made.

• Had to set up the cron job. The instructions given at MoodleDocs were to create it as the www-data user, crontab -u www-data -e 
  
  First ran it on the command-line to test that it works, running it as the www-data user with
  sudo -u www-data /usr/bin/php   /var/www/sssvv/admin/cli/cron.php
  - that took 2 minutes to run the first time, but later runs finished in a second or so.
  
  But on this Ubuntu 20.04 machine, a cron set for www-data did not work - probably as a security feature - though it did have a file in /var/spool/cron/crontabs as a comment on this page mentions. So, made a cron job as root, but running as www-data as per this post.
  * * * * * su www-data -s /bin/bash -c "/usr/bin/php /var/www/sssvv/admin/cli/cron.php"
  
  
• Had to increase the php file upload limit. 
  sudo nano /etc/php/7.4/apache2/php.ini
  ( Ctrl+w to find and change post_max_size
  Ctrl+w to find and change upload_max_filesize
  Ctrl+w to find and change max_execution_time  - set to 600 for now.)
  sudo apachectl restart
  
  
• Make the disk mount permanent by editing /etc/fstab - had to find the UUID of the partition using
  lsblk -f
  sudo blkid
  and then in /etc/fstab, added the line at the end,
  UUID=<uuid found using blkid>   /datadrive/mount/path   ext4   defaults,nofail   1   2
  Doing a umount and then a mount -a would mount it from fstab, so that we can correct errors if any. 
  
  

Linux Mint Workspace size

In order to disable extra workspaces on Linux Mint,
https://forums.linuxmint.com/viewtopic.php?t=280563

Ctrl+Alt+Up Arrow to enter the workspaces mode, select each unwanted workspace and choose to close.

Then my recurrent issue on Linux Mint installs on a Macbook - the workspace size seems to be much bigger than the screen size. Menu - Preferences - Display - found that two screens were enabled - disabled one to solve the issue.

delete a service on Windows server

One of our servers had an issue with RAM being used up. Found that it was due to a second instance of Mysqld being created as a service as seen in the screenshot. 

The two instances of Mysql were conflicting with each other, and causing mysqld to continuously use up RAM and CPU. I deleted the 2nd instance of mysqld, following this procedure.

• Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services key with regedit
• Select the key of the service you want to delete and choose Edit menu - Delete

Moodle layout - removing the footer

This page has a discussion about hiding the footer in Moodle where Boost theme is used - 
https://moodle.org/mod/forum/discuss.php?d=349625

The idea of hiding a section with something like
footer#page-footer {display: none;}
in the custom scss box is useful. 

In the Moove theme, there is the option to not display the footer, as noted elsewhere in the thread. 

removing android app splash screen

Quite an involved process to remove the splash screen, and many places where things can go wrong - 
https://stackoverflow.com/questions/48239602/how-to-remove-splash-activity-from-an-existing-project/482396

Probably just easier to customize the splash screen with our graphics, and reduce its time interval. For example,
val SPLASH_TIME_OUT = 5000
in  PYF-SmartWebView-3.5

install parse failed no certificates error for Android apk

It turns out that this is some sort of catch-all error, and various things which can cause this
Installation error: INSTALL_PARSE_FAILED_NO_CERTIFICATES
for Android apk files are discussed at
https://stackoverflow.com/questions/2914105/what-is-install-parse-failed-no-certificates-error

My issue turned out to be an incorrect signing configuration as in the top answer. 

warning in firefox for mixed content

Copy-pasting from a long explanation I had sent to someone:

The reason why Firefox shows the warnings and Chrome doesn't is because Chrome automatically checks for https links for mixed content, and automatically serves the https link instead of the http link in case it works.

You can find out which particular links are serving the mixed content (http links on a https page) by choosing inspect element when you right-click on the page, and seeing the console output.

Looks like Firefox will display the warning even if the http link is redirected to an https link - 

https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_content

"To fix this type of error, all requests to HTTP content should be removed and replaced with content served over HTTPS. Some common examples of mixed content include JavaScript files, stylesheets, images, videos, and other media.

Note: The console will display a message indicating if mixed-display content is being successfully upgraded from HTTP to HTTPS  (instead of a warning about "Loading mixed (insecure) display content")."

So even with cloudflare proxying set to strict https, Firefox would show this warning (as of March 2021).

Then, if you want this to be fixed, ask whoever has access to the web server to change all http requests to https - 

https://themify.me/blog/mass-replace-urls-https-wordpress-database

SCORM creation from Microsoft Word documents and other tools

Some discussion about SCORM - sharable content object reference model - packages, importing from Word etc - 
https://moodle.org/mod/forum/discuss.php?d=57059

sending emails as a particular Google Workspace user from Moodle

My reply to a query about a Moodle instance sending emails with a professor's email id instead of a generic address, and how to change it - 

1. Please try logging on to the-desired-email@ourdomain.tld on your browser, see if there is some issue with the username and password.

2. As you know, on moodle, you have to set up the outgoing email configuration with that username and pw at

Site administration > Server > Email > Outgoing mail configuration.

3. The sending of emails has to be authenticated and set up

https://support.google.com/a/answer/176600

4. Why the professor's account works and not the-desired-email account - probably because the professor's account has SMTP enabled from earlier. 

You could please do the setup for (3.) using the recommended option, using smtp-relay.gmail.com 

ssh keep alive

ssh kept timing out on one of our servers - when the terminal window inactive for 10 minutes or something like that. Came across this article to fix it - did it using the client side, adding the line: ServerAliveInterval 60
to the file /etc/ssh/ssh_config

failed attempts to build the moodle app apk

 Documenting my failed attempts to build the Moodle Android app.

Edit: The successful attempt is documented in this post.

For customizing
We may need to follow
http://blog.vinodsingh.com/2020/05/how-to-customize-moodle-mobile-app.html

But even the basic moodleapp fails to build - as below.
Android Studio version etc as of March 4th to 7th 2021.


Steps for build
---------------

Following https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2
except for some exceptions, and ignoring errors.

wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash




nvm install node
# don't do nvm use 11 - current version is 15, that works. 11 has bugs.

sudo apt-get install libsecret-1-dev

git clone https://github.com/moodlehq/moodleapp.git moodleapp
cd moodleapp
git checkout integration

# dont't run npm run setup - run each individually, since errors can be ignored.
npm install 
#(
#npm ERR! npm ERR! network aborted
#npm ERR! npm ERR! network This is a problem related to network connectivity.
#Re-ran ...
#)
#Took ~ 10 minutes on lenovoPC, 2 minutes on GCP.

npx cordova prepare 
# ignore the errors.
#Failed to restore plugin "phonegap-plugin-push". You might need to try adding it again. 
# Error: CordovaError: Failed #to fetch plugin 
# git+https://github.com/moodlemobile/phonegap-plugin-push.git#moodle-v3 via registry.
#Probably this is either a connection problem, or plugin spec is incorrect.
#Check your connection and plugin name/version/URL.
# (Ran again, due to internet breakage)


npx gulp

npm start
# Did not show any errors, showed waiting for connection

npx ionic cordova platform remove android
npx ionic cordova platform remove ios
npx ionic cordova platform add android
# did not do npx ionic cordova platform add ios
# ignored the error, 
# Failed to fetch platform cordova-android@^9.0.0
#Probably this is either a connection problem, or platform spec is incorrect.
# in verbose mode, 
#npm ERR! errno ENOTFOUND
#npm ERR! network request to https://registry.npmjs.org/coffeescript failed, 
#reason: getaddrinfo ENOTFOUND #registry.npmjs.org
#npm ERR! network This is a problem related to network connectivity.
# Ran again, 
# npx ionic cordova platform add android --verbose
# Platform android already exists.
# did remove and add again, then looks like it is not a connection problem, but platform spec is incorrect

# https://stackoverflow.com/questions/55965450/failed-to-fetch-platform-cordova-android8-0-0
# ignoring,

npm run dev:android
# gave error, but wrote
#cordova-android-support-gradle-release: Android platform: V7+
#cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/app/#build.gradle
#cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/
#cordova-android-support-gradle-release/moodlemobile-cordova-android-support-gradle-release.gradle
#[ERROR] An error occurred while running subprocess cordova.

# on GCP, error was No valid Android SDK root found.
# did again on a fresh terminal so that .bashrc is processed again,
# then it worked.

sudo apt-get install gradle
sudo apt-get install libgradle-android-plugin-java

Edits as per
https://docs.moodle.org/dev/Setting_up_your_development_environment_for_Moodle_Mobile_2#Compiling_using_AOT

npm run ionic:build -- --prod
# LenovoPC bogged down since it uses too much memory and goes into swap, trying on cloud dev machine.
# GCP finished in 6 minutes - used nearly all of 16 GB RAM. 200% cpu of 4 core machine.
npx cordova run android

# on GCP, gave response as below
# npx cordova run android
Conflict found, edit-config changes from config.xml will overwrite plugin.xml changes
9.0.0
cordova-android-support-gradle-release: Android platform: V7+
cordova-android-support-gradle-release: Wrote custom version '27.1.0' to 
/home/user/moodleapp/platforms/android/app/build.gradle
#cordova-android-support-gradle-release: 
#Wrote custom version '27.1.0' to 
#/home/user/moodleapp/platforms/android/cordova-android-support-gradle-release/moodlemobile-cordova-android-support-gradle-release.gradle
Cannot find module 'xcode'
Require stack:
- /home/user/moodleapp/plugins/cordova-plugin-add-swift-support/src/add-swift-support.js
- /home/user/moodleapp/node_modules/cordova-lib/src/hooks/HooksRunner.js
- /home/user/moodleapp/node_modules/cordova-lib/src/plugman/install.js
- /home/user/moodleapp/node_modules/cordova-lib/src/plugman/plugman.js
- /home/user/moodleapp/node_modules/cordova-lib/cordova-lib.js
- /home/user/moodleapp/node_modules/cordova/src/help.js
- /home/user/moodleapp/node_modules/cordova/src/cli.js
- /home/user/moodleapp/node_modules/cordova/bin/cordova

###########

# so tried importing into android studio
# https://cordova.apache.org/docs/en/10.x/guide/platforms/android/index.html#debugging
# 

#android studio gave error as follows:
# ~/androidstudioerr.txt
#Manifest merger failed : uses-sdk:minSdkVersion 19 cannot be smaller than version 22 declared in library [:CordovaLib] 
#/home/user/moodleapp/platforms/android/CordovaLib/build/intermediates/library_manifest/debug/AndroidManifest.xml 
#as the library might be using APIs not available in 19
#	Suggestion: use a compatible library with a minSdk of at most 19,
#		or increase this project's minSdk version to at least 22,
#		or use tools:overrideLibrary="org.apache.cordova" to force usage (may lead to runtime failures
#
# https://github.com/apache/cordova-android/issues/1070
# so edited config.xml with the info as given in the above, but that gave unbound prefix error.
# 
# Then, tried changing minSdkVersion=22 (from 19) on line 212 of config.xml
# still
# so, https://www.learningsomethingnew.com/how-and-why-to-change-your-android-cordova-apps-min-sdk-version
# but cordova platform rm android
# cordova: command not found.
# so tried
npx ionic cordova platform remove android
npx ionic cordova platform add android
# but the same error inside android studio.

Edit: The successful attempt is documented in this post.

German letters without diacritics

Sometimes we have to post descriptions on our schedule page of German programs, which may contain diacritics. But our schedule software currently doesn't support unicode, so I've to convert those to characters without diacritics. This page has a handy conversion guide - mostly adding e for umlauts.

REISUB - for restarting linux machines which are unresponsive

From http://blog.kember.net/articles/reisub-the-gentle-linux-restart/ - 

https://en.wikipedia.org/wiki/Magic_SysRq_key
says that on linux mint, it is Ctrl Alt PrtScr, and then - slowly - 

Alt + REISUO to shut down, REISUB to reboot.

• Switch from RAW to XLATE
• Send Sigterm
• Send Sigkill
• Sync all mounted filesystems
• remount in read-only
• Shutdown.

Running Linux Mint 20 on a Lenovo B460 laptop, on hitting Ctrl Alt F2 and seeing the 2nd terminal, 

R

E

I

show "This SysReq is disabled."

https://superuser.com/questions/375816/how-to-check-which-sysrq-functions-are-enabled

has details on the bitmask etc. 

cat /proc/sys/kernel/sysrq

"0 - disable every SysRq function.

1 - enable every SysRq function.

2 - enable control of console logging level

4 - enable control of keyboard (SAK, unraw)

8 - enable debugging dumps of processes etc.

16 - enable sync command

32 - enable remount read-only

64 - enable signalling of processes (term, kill, oom-kill)

128 - allow reboot/poweroff

256 - allow nicing of all RT tasks

438 = 2 + 4 + 16 + 32 + 128 + 256, so only the functions associated with those numbers are allowed. Read all about it in the documentation."

deploy from github to server with php

https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account

https://medium.com/riow/deploy-to-production-server-with-git-using-php-ab69b13f78ad

But then didn't use because of this,

https://medium.com/@skoskie/oh-no-c1068ed30c6b

server admin activities for an Asst. Admin

Excerpts from my reply to a colleague in a sister institution who had joined recently and wanted to get up to speed with his duties - 

These are the activities I have done to help the LMS team - 

1. Setting DNS records in Cloudflare - 

https://ns1.com/resources/dns-types-records-servers-and-queries

https://en.wikipedia.org/wiki/Cloudflare

https://support.cloudflare.com/hc/en-us/articles/360019093151-Managing-DNS-records-in-Cloudflare

The DNS records are currently handled by V----, I help him when needed.

2. Helping with load testing for servername.our.domain - R--- did most of the work - 

https://hnsws.blogspot.com/2020/12/load-testing-moodle-server-log-in-with.html

https://docs.moodle.org/310/en/Performance_recommendations#Hardware_configuration

Please see the test reports made by Prof. S----- and R-----.

3. Setting up SSL (https) using bitnami's tool - bncert-tool

https://hnsws.blogspot.com/2020/12/location-of-apache-configuration-files.html

In case the machine in question is not a bitnami VM,

https://hnsws.blogspot.com/2020/11/setting-up-letsencrypt-certificates-for.html

https://certbot.eff.org/

4. For doing the above (point 3.), you will need to know how to use the command-line, and how to use ssh - 

https://www.ucl.ac.uk/isd/what-ssh-and-how-do-i-use-it

https://en.wikipedia.org/wiki/PuTTY

https://www.chiark.greenend.org.uk/~sgtatham/putty/faq.html

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstancesLinux.html

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html

5. If you are not familiar with the Linux terminal (command-line), 

https://www.google.com/search?q=getting+started+with+the+linux+command+line

creating a backup of a web server

Process followed for creating a copy of one of our web servers on a backup server - 

• Copied the conf files using sftp.
• Created the user using adduser 
• mkdir downloads as the new user,
• added to www-data group and changed permissions as on our server,
  # chown nameofuser:www-data downloads
  # chmod 755 downloads
  
  
• Check if server2 has enough disk space before running rsync.
  Currently seems to have ~600 GB available.
  
  
• Ran a2ensite for all the missing sites, and 
• service apache2 reload
  

Edit: Now that the servers run https sites, must remember to set up suitable ACME clients as available for the respective OS, for managing LetsEncrypt certificates. 

For up-to-date Ubuntu-based Linux distros, 
sudo apt install certbot python3-certbot-apache

For older distros without support, acme.sh is what I have used in the past.

For Windows, win-acme is a good option.

certbot missing apache plugin

I too faced the same issue described in the post below, with the same solution - 
apt install python3-certbot-apache

https://community.letsencrypt.org/t/certbot-missing-apache-plugin/58579

Moodle directory permissions

"Invalid permissions detected when trying to create a directory. Turn debugging on for further details." 

Simple fix would be:

sudo chown -R  www-data:www-data /var/www/our_moodle_dir
sudo chown -R  www-data:www-data /var/www/our_moodle_files_dir

mount a new data disk on an Azure Linux VM

https://blog.e-zest.com/how-to-create-attach-and-mount-a-disk-to-linux-vm-microsoft-azure

Basically create the disk from the Azure portal, then

• find the name of the device using dmesg - for eg. /dev/sdc
• sudo fdisk /dev/sdc
• sudo mkfs -t ext4 /dev/sdc1
• sudo mkdir DriveFolder; sudo mount /dev/sdc1 DriveFolder
• append to /etc/fstab using sudo blkid to find the UUID of the disk. Eg. 
  UUID=33333333-3b3b-3c3c-3d3d-3e3e3e3e3e3e   /datadrive   ext4   defaults,nofail   1   2
  

https://docs.microsoft.com/en-us/azure/virtual-machines/linux/attach-disk-portal
has a better set of instructions. 

mounting Azure blob storage

We looked at the different ways in which Azure storage could be mounted on a Linux Moodle instance, using blobfuse

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-how-to-mount-container-linux

but did not find any mentions using

mount

or

crontab -l (for various users)

Edit: It turned out it was using Moodle's Object storage file system plugin, which manages all the storage from within Moodle. But see this caveat.

Fresh install of Moodle on Ubuntu 20.04 with apt

Using the command at https://www.tecmint.com/install-moodle-in-ubuntu/

sudo apt update
sudo apt install php-common php-iconv php-curl php-mbstring php-xmlrpc php-soap php-zip php-gd php-xml php-intl php-json libpcre3 libpcre3-dev graphviz aspell ghostscript clamav

Then git-based install after

sudo apt install git

Edit: For the error "PHP has not been properly configured with the MySQLi extension for it to communicate with MySQL. Please check your php.ini file or recompile PHP."
we can follow the installation instructions at
https://docs.moodle.org/310/en/Step-by-step_Installation_Guide_for_Ubuntu

Installing the lapp stack on Ubuntu 20

https://kb.amijani.net/web-server/how-to-install-lapp-apache-php-postgresql-on-ubuntu-20-04/

Basically, 
sudo apt install apache2
sudo apt install php7.4 libapache2-mod-php7.4 openssl php-imagick php7.4-common php7.4-curl php7.4-gd php7.4-imap php7.4-intl php7.4-json php7.4-ldap php7.4-mbstring php7.4-pgsql php-ssh2 php7.4-xml php7.4-zip unzip

Then install PostgreSQL with
https://www.digitalocean.com/community/tutorials/how-to-install-postgresql-on-ubuntu-20-04-quickstart

connecting to a pptp VPN

A VPN to some servers was configured on Windows 10 clients as 
Type of sign-in info: User name and password
and under "Advanced", 
VPN type: Automatic.

Other windows clients were able to log on to the VPN with no problems, but when I tried with Linux Mint and the default pptp VPN settings, the connection was failing. My first thought was that some setting was missing, but it turned out that the reason for the VPN to fail was that my internet connection was via a mobile hotspot - just connecting via a broadband connection did the trick, I did not have to do any port-forwarding or any settings on the router. And then I found that some mobile hotspots work too - 
LG Q6 with Airtel 4G on primary SIM slot - no
LG Q6 with Airtel 3G on primary SIM slot - sometimes works. 
Redmi 4 with Jio SIM - works
Realme U1 with Jio SIM in secondary SIM slot - no
JioFi router (2018 model) - works
BSNL ADSL broadband with wifi router - works

With the default routing, all connections are routed through the VPN, and normal internet browsing doesn't work. With the help of this post, changed the routing so that only connections to those machines which are on the VPN are routed through the VPN.

/var/log/syslog gave the internal gateway info, which is on the private subnet, different from the external gateway which is on a public subnet.

 

making a rectangle in Gimp

1. Select using Rectangle Select Tool
2. Edit - Stroke Selection

rsync, screen and ssh-agent

On some of our other Ubuntu servers, rsync in a cron job as root works fine with ssh and key-based authentication. But not on a CentOS server I recently logged on to, because ssh-agent was not automatically started on login.

So, I have to run the following - 

eval `ssh-agent -s` 
ssh-add /path/to/keyfile.pem

and then do the ssh -i keyfile user@remoteserver

If I want to run a long rsync job from within screen, I need to run the above commands again from within screen, or else the ssh-agent is not accessible from within screen.

The next issue was the use of rsync with directories which had spaces in them. Instead of laboriously escaping each space with a backslash, I used the --protect-args method.

rsync -azvhrt --protect-args -e ssh  "/local path/with spaces/directory to copy" "user@remotemachine.tld:/remote/path/with spaces/"

php oauth server - link dump

PHP 5

https://www.sitepoint.com/creating-a-php-oauth-server/

https://bshaffer.github.io/oauth2-server-php-docs/

PHP 7

https://github.com/thephpleague/oauth2-server

Using firebase (PHP 7)

https://github.com/jeromegamez/firebase-php-examples

Edit: We finally did not use any of these, and instead authenticated using google sign-in against our own database as mentioned at https://hnsws.blogspot.com/2021/12/using-google-sign-in-for-our-php-portal.html

creating rectangular boxes with Gimp

Just to remember - rectangular marquee to select - shortcut key is R
Then, menu item Edit - Stroke selection

Line width of 4 pixels looks quite good for onscreen viewing, like below. 

forcing chrome to reconnect to a website

For purposes like profiling site load speeds etc, making Chrome close connection to a web server and start over - 

https://stackoverflow.com/questions/37170812/force-chrome-to-close-re-open-all-tcp-tls-connections-when-profiling-with-the-ne

Go to

chrome://net-internals#sockets

and close idle sockets.

Also works for sessions opened with SSH tunnels, when we want to close the connection immediately.

HTTP to HTTPS redirection on Windows Server and IIS

Certbot can automatically add http to https redirection when used with Apache on Linux, by appending

RewriteEngine on
RewriteCond %{SERVER_NAME} =whatever.tld
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

To do the same on Windows, I followed the method given at
https://www.ssl.com/how-to/redirect-http-to-https-with-windows-iis-10/

for redirecting http to https on IIS+Windows. 

• Download and install the URL Rewrite module
• In IIS Manager, choose the relevant website, double-click URL Rewrite
• Click Add Rules on RHS, create a Blank Rule in inbound section.
• Choose Requested URL: Matches Pattern and Using: Regular Expression, with the Pattern: (.*) and Ignore Case ticked. 
• Logical Grouping: Match All, then Add...
• In the Add Condition box, Condition Input: {HTTPS}, Check if input string: Matches the Pattern, Pattern: ^OFF$, Ignore case: ticked.
• In the Action setting, Edit Inbound Rule, set Action type: Redirect, Rewrite URL: https://{HTTP_HOST}/{REQUEST_URI} ,  Append query string: uncheck, Redirect type:  Permanent (301).
• Finally, click Apply in the Actions section on Right-hand side.

This would create a web.config in the site's home directory, with contents like:

<?xml version="1.0" encoding="UTF-8"?>

<configuration>

    <system.webServer>

        <rewrite>

            <rules>

                <rule name="HTTPS Redirect" stopProcessing="true">

                    <match url="(.*)" />

                    <conditions>

                        <add input="{HTTPS}" pattern="^OFF$" />

                    </conditions>

                    <action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" appendQueryString="false" />

                </rule>

            </rules>

        </rewrite>

    </system.webServer>

</configuration>

scripts to alert admin when hard disk is full, and to delete older files

Proposed solution to alert admin when a server hard disk becomes close to full, and for auto deleting old files - 

1. Script to email us when hard disk becomes full -
https://www.google.com/search?q=script+to+email+me+when+hard+disk+crosses+free
gives this script,
https://www.linuxjournal.com/content/tech-tip-send-email-alert-when-your-disk-space-gets-low

by adjusting the THRESHOLD=90 line, we can adjust when the script will email us. 10 GB out of 500 GB hard disk means we need to put THRESHOLD=98

#!/bin/bash
CURRENT=$(df / | grep / | awk '{ print $5}' | sed 's/%//g')
THRESHOLD=98
if [ "$CURRENT" -gt "$THRESHOLD" ] ; then
    mail -s 'Disk Space Alert' mailid@domainname.com << EOF
Your root partition remaining free space is critically low. Used: $CURRENT%
EOF
fi

And a cron to run this daily like 
@daily ~/ourScriptName.sh

2. Script to automatically delete files older than one month - 

https://www.google.com/search?q=script+to+automatically+delete+files+older+than+one+month
gives us this,
https://tecadmin.net/delete-files-older-x-days/
which lists the steps manually. 

If we just write this as a script and put a cron job like the earlier script, this should work. Something like
find /our/path -name "*.zip" -type f -mtime +30 -delete

error on booting Linux Mint 20

https://www.google.com/search?q=Buffer+I%2FO+error+on+dev+sda7%2C+logical+block+32495328%2C+async+page+read

https://www.linuxquestions.org/questions/linux-hardware-18/buffer-i-o-error-on-dev-sdb1-async-page-read-4175600715/

Hardware error might be fixed by zeroing out the partition and then reformatting, the last posts above indicate.

upgrade Linux Mint 18.3 to 20

Tried the upgrade as per
https://community.linuxmint.com/tutorial/view/2416

Regenerating fonts cache ... failed.

(Probably this was due to Google Chrome or Google Fonts which were installed?)

So, just installing 20.1 as a fresh install on lenovoPC.

icecast reload

Tried an implementation of icecast compiled with ssl support - but needs reload whenever the certificate is changed.

(Edit - see the later implementation using icecast-kh which does not have this reload requirement. )

Tried
kill -HUP  processid

after
ps -A | grep icecast

to find process id.

But that did not seem to work - neither as non-root user nor as root. probably because the process was started as nohup.

So, did

kill processid

Verified with
ps -A | grep icecast

that the process was killed,

then did

nohup /path/bin/icecast -c /path/icecast_ssl.xml > sh.out 2> sh.err < /dev/null &

This seems to work fine even though the order of the certificate is reversed - cer first and priv second as compared to the other way for the self-signed cert.
Edit - this followed the cat strategy at
https://forum.armbian.com/topic/11436-solved-icecast2-and-ssl/

And again, seems to work fine even though it has RSA PRIVATE KEY instead of PRIVATE KEY as in the self-signed certificate.

Edit - see the later implementation using icecast-kh which does not have this reload requirement. 

too many authentication failures with ssh

The reason for the error "too many authentication failures" was the presence of too many ssh keys in my .ssh directory and the solution was to add the option IdentitiesOnly like

ssh -o "IdentitiesOnly=true" -i key.pem user@server.tld

or for password-based login, can also use

ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@server.tld

monitoring icecast stream with uptimerobot

Just a reminder to myself that the uptimerobot monitor for Icecast streams has to be a GET monitor, because Icecast sends a 400 Bad request error for HEAD requests. So, for example, we can set up a GET monitor for
https://streamingurl.tld/mountpoint/status.xsl

But then we would not really know if the streams are running - for that, we rely on our shell scripts to email us like
COULD NOT FIND PLAYLIST nameof.playlist AT Sun May 28 06:01:01 IST 2017
PLAYING EMERGENCY INSTRUMENTAL PLAYLIST!!

another domain name for the same page, implemented with Plesk

We wanted the same content as oldname.tld/url to be served from newname.tld/url for a domain which was being served from a shared server running Plesk. We could not do it by adding a domain alias, instead we had to do it by adding newname.tld as a new domain in Plesk and pointing it to the same directory as oldname.tld for wwwroot.

creating a Linux filesystem on a VHD file

In order to create a timeshift backup before upgrade, creating a virtual hard disk volume,
https://www.tecmint.com/create-virtual-harddisk-volume-in-linux/

This took 20 minutes on lenovo pc for 30 GB. mkfs finished in just seconds.

fiber link tests

Our fiber link, which used a 2-core media converter, had gone down. Checking various segments which were looped at different locations, each segment by itself was OK. But with all the loopings, though light was seen when testing with a "visual fault locator" or laser, one core was less bright. So, used a single-core media converter, and link became OK. 

Google Forms for creating a feedback form

Initially, I thought our non-profit edition of Google Workspace did not have Google Forms, and so created a form manually, following

https://dev.to/omerlahav/submit-a-form-to-a-google-spreadsheet-1bia

https://github.com/jamiewilson/form-to-google-sheets

and a template from jotform.com

Then PB pointed out that accessing Google Forms from Google Drive -> New -> Google Form works, and that the scripts.google link has a problem of not displaying if the user is logged on to more than one google account at the time of clicking that link.

So, PB recreated as a Google Form. 

google apps scripts limitation on multiple logins

One of the limitations of Google Apps Scripts - apps created with script.google.com fail when logged in to multiple accounts.
https://www.demandsage.com/blog/google-multi-account-authentication-bug-what-it-is-possible-workarounds/

For standalone scripts which have a url like https://script.google.com/macros/s/LONGID/exec?parameter the error shown is:
"Sorry, unable to open the file at present.

Please check the address and try again."

renewing a dot tk domain

Freenom is the domain registrar for .tk and currently, their process for renewing free domains is - 

• we get a reminder email 2 weeks before expiry
• free renewal is possible only 2 weeks before expiry
• the current link to renewal is via the Services menu item - Services -> Renew domains and not via "My Domains" as mentioned in their email. 

exploring creating issues on github using email

Checked out ifttt email -> github - that needs the email to be sent from a particular email id, which will then create an issue on github. Could possibly implement using gmail's auto-forwarding features.

https://ifttt.com/connect/email/github
https://ifttt.com/applets/TKuX3JY6-create-github-issue-via-email

Edit - Another way to do it via the Fire bot - https://www.ilovefreesoftware.com/25/programming/how-to-create-github-issues-by-sending-email.html

adding SSL (https) to http mp3 streams - Apache reverse-proxy and letsencrypt

Our audio streams were getting blocked by Chrome's insistence on https. Our initial workaround was to not use any javascript-based player, and instead just use a link to the stream URL with target = _blank to open it in a new window or tab. Now, working towards https support for our audio streams, we found two options.

1. Icecast 2.4.4 has support for SSL, but we need to recompile it on most platforms with the flag set as the icecast package which is available at this repo doesn't support SSL. ./configure --with-curl --with-openssl

2. The other option is to use an SSL reverse-proxy - Apache or Nginx.

Since we had Apache running on our server already, I tried the reverse-proxy method. I had to use acme.sh and not certbot since our server needs an OS upgrade. After the acme.sh installation, have to log out and log in (or open another shell) for the .bashrc changes to take effect. On the server, the main steps were:

a2enmod proxy
a2enmod proxy-http  #(without this, got Internal Server Error)
service apache2 restart

a2ensite stream.ourdomain.tld
service apache2 reload

The file created for stream.ourdomain.tld in /etc/apache2/sites-available/stream.ourdomain.tld.conf was copied from another virtual host and then I added the lines like

ProxyPass /nameofstream http://11.22.33.44:8567/
ProxyPassReverse /nameofstream http://11.22.33.44:8567/ 

ProxyPass /nameofanotherstream http://11.22.33.44:8577/ 
ProxyPassReverse /nameofanotherstream http://11.22.33.44:8577/ 
</VirtualHost>

The file created for the ssl version had to be hand-written since acme.sh doesn't automatically create the config files unlike certbot. stream.ourdomain.tld-ssl.conf had the additional lines

<IfModule mod_ssl.c>
<VirtualHost *:443>

SSLEngine on
  SSLCertificateFile      /etc/ssl/certs/stream.ourdomain.tld.cer
  SSLCertificateKeyFile /etc/ssl/private/stream.ourdomain.tld.key

So, with acme.sh, first issued the certificate in Apache mode as root - 

acme.sh --issue --apache -d stream.ourdomain.tld

and then installed it using

acme.sh --install-cert -d stream.ourdomain.tld \
--cert-file      /etc/ssl/certs/stream.ourdomain.tld.cer  \
--key-file       /etc/ssl/private/stream.ourdomain.tld.key

a2ensite stream.ourdomain.tld-ssl
service apache2 reload

Worked.

Everything seems to be fine - Apache was not having any issues I could detect with top and a tail -f of the Apache access log. Cloudflare proxying had been turned on, but I'm not sure if it had any impact since this was a stream and not a static file which could have been cached.

CPU utilization and RAM utilization didn't change more than 1-2% - 97% idle CPU, and all the Apache processes together were using only 14% RAM or less of our 8 GB server, which didn't increase substantially from what it was before the test. Some 300+ clients connected via our website with the new links to the various streams in the last 90 minutes or so of the test. 

Edit: See the later post, implemented with icecast-kh
https://hnsws.blogspot.com/2021/06/adding-ssl-https-to-http-mp3-streams.html

github phasing out password authentication for commits

While making some command-line commits to github, I'd received an email from github that password-based authentication is deprecated, and would be unavailable after Aug 2021. Github docs also mention that pw auth is deprecated. This stackoverflow post also discusses the change, and its advantages. So, basically we need to create the password token, give it suitable permissions and use that instead of the password on the commandline.

centre-click for restoring xfce minimized windows

Working with xfce on a GCP cloud server, minimized windows were not visible. My workaround as suggested by this forum post was to centre-click on the desktop (right+left click on my trackpad) and choose the relevant window from the list.