Mostly work related stuff which I would've entered into my "Log book". Instead of hosting it on an intranet site, outsourcing the hosting to blogger!
Friday, September 30, 2022
Zoho mail issues
Thursday, September 29, 2022
site traffic report from cloudflare
Cloudflare claims to give reports without having to install server-side code, and claims to not track individual users. Our sites' report says, data from 14 of our Cloudflare sites during the month of August
- 14 sites
- Cloudflare served 2.58 TB of data
- 70.9% Data transfer saved
- 19.38k firewall mitigated events
Wednesday, September 21, 2022
setting up IIS with dot NET core on Windows 10 - but Linux is better!
Setting up IIS and dot NET core for Windows 10 - this may be slightly different for Windows 11 and or Windows Server OSes.
1. Installing IIS - Control Panel - Programs and Features - Turn Windows Features on or off - choose IIS, IIS manager, (optionally open them up and select all the features like FTP and so on if needed).
2. Installing the dot NET Hosting bundle and deployment - First download and install dot NET core hosting bundle - googling found the updated link here - https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-aspnetcore-6.0.8-windows-hosting-bundle-installer. Then "publish to folder" in Visual Studio. Then copy to the relevant folder inside the relevant IIS website.
3. Create application in IIS - (a) create a new IIS Application Pool under the .NET CLR version of “No Managed Code”.
(b) choose Add Application for the relevant website, use the Application Pool listed above, and the folder to which the deployed app was copied
(c) and now if the website is enabled, the app should be working.
Apparently, we can try and run dot NET web apps on Linux also,
https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#1804-
We can run console app with
dotnet myApp.dll
or we can publish a standalone version with
dotnet publish -c release -r ubuntu.16.04-x64 --self-contained
and run it after copying it to the Ubuntu server with
chmod +x ./appname
./appname
as in the post below.
https://stackoverflow.com/questions/46843863/how-to-run-a-net-core-console-application-on-linux
Complete howto on deploying and running on an Ubuntu server -
https://www.c-sharpcorner.com/article/how-to-deploy-net-core-application-on-linux/
Apparently, there is a nearly 50% performance boost by running on Linux!
https://stackoverflow.com/questions/44334125/net-core-on-windows-vs-linux
https://robertoprevato.github.io/More-about-Linux-vs-Windows-hosted-ASP-NET-Core-applications-in-Azure-App-Service/
Tuesday, September 20, 2022
deleting an Azure free trial and deleting a hotmail account
Earlier, I had posted about stopping the Azure free trial from an unmanaged domain account. This post is about stopping an Azure free trial subscription and deleting a free hotmail account.
https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/cancel-azure-subscription
portal.azure.com --> Cost Management + Billing --> Overview --> your plan --> select Cancel --> verify that you want to cancel and select Yes, cancel.
After that, to delete the hotmail account which had that free trial,
https://www.makeuseof.com/tag/how-to-delete-outlook-hotmail-account/
account.microsoft.com --> login --> Your Info tab --> Scroll down to the Help with Microsoft account --> How to close your account --> Choose whether you want Microsoft to retain your data for 30 days or 60 days --> Next etc (various security confirmations)
This info is probably subject to change as these portals constantly undergo slight revisions in UI.
Monday, September 19, 2022
steps taken for a suspended Google Workspace domain
Unfortunately, one of our domains missed out migration to non-profit edition. Copy-pasting from some email exchanges which happened, to document one possible way of dealing with such a situation.
The following were carried out:
1. For emergency continuation of requireduser@our-lapsed-domain.tld, our-lapsed-domain.tld was upgraded to Business starter. Would cost approx Rs. 250 per month, so instead of continuing this, we would migrate the users.
2. For doing this, the easiest way was to find out the admin account of our-lapsed-domain.tld in the legacy Google Apps for Domains.
Initially, many of our domains had been administered by Mr. MM's team, so I tried searching for their hand-over email which listed all assets and all logins, in which I had also been cc'd - found our-lapsed-domain.tld also in that list. Tried with its administrative account - apparently not logged in for many years - and enrolled in Business Starter.
There were 4 accounts seen - of which only the administrative account and requireduser@our-lapsed-domain.tld had any data.
3. Have started the migration of archived emails from supersededemail@our-lapsed-domain.tld (just for safety's sake, probably these have already been saved earlier) to currentemail@anotherdomain.org
4. Will delete notused@our-lapsed-domain.tld - since it seems to be not in use.
5. Since the current email id used by Dr. is in ourdomain.org, P has been requested to ask S to create requireduser@ourdomain.org. Once this is done, requireduser@our-lapsed-domain.tld will be migrated there - perhaps an outage of a day, which can be decided by requireduser at their convenience. Outage will be only in terms of accessing old emails. Fresh emails will go to requireduser@our-lapsed-domain.tld without any outage.
6. Once the new email id is ready, we can start forwarding new email which goes to requireduser@our-lapsed-domain.tld to requireduser@ourdomain.org - for this, I have used https://improvmx.com/
in the past, and seems to be a good service.
I can set up forwarding for supersededemail@our-lapsed-domain.tld to currentemail@anotherdomain.org also if desired. And adminaccount to orgadminaccount@gmail if desired.
7. Future - checking if there are any other domains which have been orphaned like this - I will check
(a) the hand-over email from MM's team
(b) the MX records in all domains which are under our control
and make a list of all domains which have emails enabled, and who administers them.
----
Later, to S ...
---
----
Later, after S had created the new account ...
---
In case you use any other services - google keep, youtube, blogger, calendar, drive, anything else - please take a backup by logging on to requireduser@our-lapsed-domain.tld and then navigating to takeout.google.com
This is important, so please let me know once you have taken the backup (or let me know if you don't use any services except email.)
So, from 2 pm tomorrow or so, you would not have access to requireduser@our-lapsed-domain.tld
The emails will start getting migrated hopefully by 2:30 pm.
Depending on the total disk size as well as the number of emails, this process can take between 1-7 days.
(The migration happens via IMAP,
https://support.google.com/a/answer/9476255?hl=en
In our experience, if total emails size is less than 2 GB, it would take only one day.)
You will start seeing the newest emails - from the last few weeks - within an hour, and the older emails will continue to be added to requireduser@ourdomain.org over the course of the migration.
direct url of play console and google apps script editor url
For uploading or editing apps in the Google Play Store, the play console url, if logged in to multiple accounts at the same time, would be like
https://play.google.com/console/u/6/
(if using the account number 6, etc.)
similar to
https://drive.google.com/drive/u/6 etc.
The google apps script editor url is of the form
https://script.google.com/u/6/home/projects/project-id/edit
This usually works if we just double-click on the script in google drive if the editor is set to "new editor", but if the editor is set to the "classic" version of GA script editor, double-clicking on a script may not work, we need to use the above url.
Thursday, September 15, 2022
xfce icons still not showing
Running xfce over VNC on a cloud server, the icons were not showing up, even after
# gtk-update-icon-cache --force /usr/share/icons/hicolor
Tuesday, September 13, 2022
modify and build the Moodle app version 4+
Edit - There is a newer post on building the release version entirely from Github Actions - for the moodle app v4.1.0.
There are a few changes from my earlier post about building the Moodle app. I'll try to write this post as a bash script which could be run on an Ubuntu-based machine which does not have the build environment set up, so that it would be possible to use something like Travis CI to build it. The current script below will not work without a gui on headless servers, since there are some steps like previewing in Chromium, installing android studio etc require a GUI.
# the following are for a headless server, to get a gui over VNC
# https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-ubuntu-18-04
sudo apt update
sudo apt install xfce4 xfce4-goodies -y
sudo apt install tightvncserver -y
vncserver
# This starts up VNC server and asks to set up a password.
# The following will be after logging on to VNC - my preferred method is
# to tunnel port 5901 over ssh.
# following the documentation at https://moodledev.io/general/app/development/setup
sudo apt install git chromium-browser -y
# from https://github.com/nvm-sh/nvm#installing-and-updating
wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
source ~/.bashrc
git clone https://github.com/moodlehq/moodleapp.git
cd moodleapp
git checkout v4.0.1
# Here we make all the customizations, described at the bottom of this post
cd ..
git clone https://github.com/ourPrivate/Repo-with-customization-files.git
cd customizedMoodleApp4changes
cp -Rvf * ../moodleapp
cd ../moodleapp
# node version should be < v15,
# https://github.com/moodlehq/moodleapp/blob/master/package.json#L178
nvm install 14.15.0
npm install
npm start
# npm start is to see if it works in chromium browser
# Initially it will open the browser and complain connection refused.
# It will take around 5 minutes to compile, only then a browser reload will work.
# Even after the first compile, it takes around 90 seconds on a 16 GB machine for
# each npm start command - till the chokidar error comes up.
# Fix the chokidar error - too many watch files - with
# echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
# Check if this is OK with
# cat /proc/sys/fs/inotify/max_user_watches
# For building Android or iOS app, need more setup,
# https://moodledev.io/general/app/development/setup
# via my previous post, need to install cordova -
npm install cordova
npm i -g cordova-res
sudo apt install libsecret-1-dev -y
sudo apt install openjdk-11-jdk -y
sudo add-apt-repository ppa:maarten-fonville/android-studio
sudo apt update
sudo apt install android-studio -y
# This ppa install does not add the required environment variables, so
# to find java home, via https://www.baeldung.com/find-java-home
dirname $(dirname $(readlink -f $(which javac)))
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH
export ANDROID_SDK_ROOT=/home/azureuser/Android/Sdk
export PATH=$ANDROID_SDK_ROOT/tools:$ANDROID_SDK_ROOT/tools/bin:$ANDROID_SDK_ROOT/platform-tools:$PATH
# We may need to install the android-30 SDK separately,
# https://developer.android.com/studio/command-line/sdkmanager
# but it is far easier to do it from within the gui SDK manager
npm run prod:android
# This will probably fail due to not finding gradle,
# and at this point, we can just open android studio,
# import the platforms/android folder, and build there.
# The android studio build may complain that build tools ver 30 are not found.
# The way to install them using SDK manager is to choose
# the SDK Tools tab, and check the "Show package details"
# in order to make the Build tools 30.0.3 show up, which we can mark and install.
# Unfortunately, building this way with the above steps seems to result in
# an app with the default cordova or ionic icons. To change to our icons,
# we could use Android Studio's Image Asset Studio
# https://github.com/ionic-team/capacitor-assets/issues/108
# But the standard way to generate icons as per
# https://capacitorjs.com/docs/guides/splash-screens-and-icons
# was creating files in android directory instead of platforms/android, so
# since mv cannot do it,
cordova-res android --skip-config --copy
cp -Rvf android/* platforms/android
# https://stackoverflow.com/questions/37300811/android-studio-dev-kvm-device-permission-denied
# if you want to install an Android emulator
# sudo apt install qemu-kvm
#sudo adduser $USER kvm
# sudo reboot
# To run the emulator outside Android Studio in a separate window,
# start the emulator before opening the project
# https://stackoverflow.com/questions/70986530/android-studio-emulator-in-a-separate-window
To sign the app with our existing keystore, followed a procedure similar to my previous post, with a change that the location of the Build types tab has changed in the latest 2021.2.1 chipmunk version of Android Studio. In short, following the official documentation (which shows screenshots of the older version)
- Project window --> right-click on app --> Open Module settings
- Modules on LHS pane --> app --> Signing configs tab --> create a new signing config by giving it some name like releaseconfig
- Modules --> app --> Build types tab --> choose the newly created signing config under "Signing" for the release build. (Extreme scrolling to the right was needed before the drop-down was visible, in one case)
- Build variants --> choose the release build
The app build so far was fine. But unfortunately, the 4.00 branch of the Moodle app has show-stopping bugs -
https://tracker.moodle.org/browse/MOBILE-4135
https://tracker.moodle.org/browse/MOBILE-4084
So, I'll continue trying fixes for these in separate posts. One option might be to use the 3.9.4 or 3.9.5 version and fix only the zip path traversal vulnerability in them -
https://tracker.moodle.org/browse/MOBILE-3949
Virus and threat protection not opening in Windows 10
After restoring Windows 10 on a refurbished mini-desktop purchased on Amazon, found that the Start button was not displaying any content when left-clicked, and the Virus / threat protection / Windows Defender settings page was also not opening. Tried these steps, https://www.thewindowsclub.com/virus-and-threat-protection-not-working-on-windows
- cmd as administrator using right-click of start menu
- ran System File checker with
sfc /scannow
- this showed some errors, and said they were fixed. - ran Deployment Image Servicing and Management with
DISM.exe /Online /Cleanup-image /Restorehealth
It also did not report any issues, but the problem continued. Then tried anti-rootkit scan from Malwarebytes - it found and fixed some issues. Rebooted, ran another malwarebytes full scan, did windows update. Then Windows Defender and Virus threat protection page started opening. Start menu (left click) still doesn't work, but other things seem ok.
Make Windows boot by Default in Windows + Linux Mint Dual Boot System
Via https://www.techmesto.com/set-windows-as-default-in-linux-dual-boot/
sudo nano /etc/default/grub
and edit the line which says
GRUB_DEFAULT=2
(the entries start from 0, up to down.)
Monday, September 12, 2022
hotstar scaling
Happened to come across this video - Scaling hotstar.com for 25 million concurrent viewers - interesting to note the types of challenges when scaling up with record-breaking numbers of users. And there's a whole channel with such tech talks, HasGeekTV
Friday, September 09, 2022
Personal finance apps - UI - SIP delete in Groww and Valueresearchonline
In the Groww app,
--> Mutual funds tab (at the bottom)
--> Dashboard tab (at the top)
--> SIP is shown on top - click on the arrow to the right
click on the relevant sip's arrow to the right
Cancel SIP button on top.
In valueresearchonline.com's portfolio manager,
--> My investments
--> Overview
--> Action drop-down - last column on the right
--> Edit/view SIP / SWP etc
Wednesday, September 07, 2022
Samsung Galaxy Book 2 - Samsung settings on Windows
Some extra settings are available for the Samsung Galaxy Book2 via Start - Samsung Settings -
https://www.youtube.com/watch?v=QOVJK-yaCpo
I have turned on the "Power on when lid opened" setting, and the "Charge only up to 85% to conserve battery" setting.
Monday, September 05, 2022
Moodle page and requiring login
We can create a new custom page in a Moodle site by following the template below,
<?php
require_once(dirname(__FILE__) . '/../config.php');
$PAGE->set_context(get_context_instance(CONTEXT_SYSTEM));
$PAGE->set_url('/ourfolder/ourfile.php');
echo $OUTPUT->header();
?>
<p>Our content</p>
<?php
echo $OUTPUT->footer();
?>
Another page example is at
https://moodledev.io/docs/apis/subsystems/output
Then for requiring login, we can add require_login() and then use the conditional isguestuser() for allowing or not allowing guest users.
https://docs.moodle.org/dev/Access_API
https://moodledev.io/docs/apis/subsystems/access
<?php
require_once(dirname(__FILE__) . '/../config.php');
$PAGE->set_context(get_context_instance(CONTEXT_SYSTEM));
$PAGE->set_url('/ourfolder/ourfile.php');
$PAGE->set_title('Our page title');
$PAGE->set_pagelayout('standard');
require_login();
echo $OUTPUT->header();
if (isguestuser()) {
echo "<h2>Please log in to see this page!</h2>";
}
else {
?>
<iframe
src="https://our.iframe.source.pg/"
name="search_result_frame" width="100%" height="800px">
</iframe>
<?php
}
echo $OUTPUT->footer();
?>
Then we can add this newly created page to our Moodle navigation using various methods. One way would be to add it to the custom menu settings box,
https://oursite.org/admin/settings.php?section=themesettings
and then also adding it to the custom menu items on the mobile app,
https://oursite.org/admin/settings.php?section=mobilefeatures
Edit: Our page worked only in the "open in browser" mode because it had javascript and a form inside an iframe, and we had not included this page in the app code. Simple pages would probably work in the in-app browser mode also.
Sunday, September 04, 2022
creating zip file of Google Drive folder - Google Apps Script
Another way of handling a google drive folder, instead of directly giving a link to it from our Google apps script web application, might be to zip it,
Saturday, September 03, 2022
cloudflare and 3rd level domains
investigating Linux server crash
One of our Linux servers stopped responding unexpectedly. Checking for issues - https://serverfault.com/questions/386985/how-to-investigate-unexpected-linux-server-shut-down
grep -i error /var/log/syslog
Sep 2 01:55:49 servername-php systemd[1]: Condition check resulted in Process error reports when automatic reporting is enabled (file watch) being skipped.
Did not check
/var/log/apache2/error.log
Instead, checked for ssh brute-forcing and or other issues with the logs, but did not find anything just before the shutdown.
https://serverfault.com/questions/68500/how-do-i-determine-if-my-linux-box-has-been-infiltrated
tail /var/log/auth.log -n4000 | more
Lots of Unable to negotiate and Invalid user ssh brute forcing attempts, but nothing at the relevant time. But did find something which pointed the finger to some process running on the server -
Sep 1 18:42:40 servername-php su: pam_unix(su:session): session opened for user www-data by (uid=0)
Sep 1 18:42:40 servername-php su: pam_systemd(su:session): Failed to create session: Connection timed out
Then, checked the Moodle logs on that machine, https://oursite.tld/report/log/
Found that only very few users were logged in (if at all). So, not due to too many users logged in.
https://oursite.tld/admin/tasklogs.php
Filter result = Fail
Date range 1 to 3 - lots of email fails.
Execute adhoc task: mod_forum\task\send_user_notifications
... started 08:36:04. Current memory use 41.1 MB.
Sending messages to ..retracted.. (2404)
Error: lib/moodlelib.php email_to_user(): SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/TroubleshootingSMTP server error: Called RSET without being connected
Failed to send post 3878
Error: lib/moodlelib.php email_to_user(): SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/TroubleshootingSMTP server error: Called RSET without being connected
Failed to send post 3879
Sent 0 messages with 2 failures
... used 44 dbqueries
... used 1.9203059673309 seconds
Adhoc task failed: mod_forum\task\send_user_notifications,error/Error sending posts.
... very large number of Email sending failed notifications.
44,000 failed notifications between 11:45 pm and 12:45 pm last night.
Most probably this is the reason for the server failure.
The error says Called RSET without being connected
which is usually due to connection issues with the hosting provider or mail server.
Azure or Google might be blocking excessively rapid email sending.
The site default was to send an email notification to every user who has posted to a forum whenever anyone else posts to a forum. I have now changed this to "email daily digest" instead.
With the earlier setting, if 100 users post to a forum (like there are forums which say "please post your feedback", so all enrolled users may try to post there)
then around 5000 emails would go out. (When user 2 posts, 1 email is sent to user 1. When user 3 posts, 2 emails are sent, to 1&2. When user 4 posts, 3 emails are sent, to 1,2,3 - and so on.)
But changing this setting alone is probably not sufficient. This is only the default setting for new users and new forums, I think. We would have to manually change the messaging preferences for all existing users to prevent such email storms. I'm not sure how to do this. Maybe by directly modifying the database? To be checked out. There is some discussion about this at
https://moodle.org/mod/forum/discuss.php?d=28908
which I have to read and understand.
The way Moodle is designed, individual users can set their preference, whether they want an email or digest or not. There doesn't seem to be an admin setting to reset preferences for all users.
There are many other places where emails are sent in Moodle. The above two are just indicative.
Checking the database, for a user login which I control, maildigest=0, mailstop=0 - mailstop=1 would stop all emails I guess.
maildigest=1 for all users might be a bit draconian?
A discussion about this is at https://moodle.org/mod/forum/discuss.php?d=28908
But according to https://moodle.org/mod/forum/discuss.php?d=19599
changing maildigest=1 will not change anything for current users.
creating an efi partition manually in Linux
For system recovery, when the EFI partition is hopelessly damaged or over-written, we can try re-creating an efi partition, installing refind in it or reinstalling Windows or something like that.
https://askubuntu.com/questions/743095/how-to-prepare-a-disk-on-an-efi-based-pc-for-ubuntu
Fat 32, 1st partition, after creating, which should be 128 MB minimum, but recommended 512 MB, add flags boot and esp.
Friday, September 02, 2022
how to create an EFI boot partition
- create a fat32 1st primary partition of size 128 MB or 256 MB or something like that (needs to be first or at least within the first 2.2 TB only if the disk is > 2.2 TB)
- after creating the partition, add the flags boot and esp
Pushbullet and uptimerobot
Edwiser reports for Moodle
adding display modes in Linux
Thursday, September 01, 2022
win-acme - scheduled update problem and solution
There was a notification from one of our Windows servers that the LetsEncrypt certificates would expire in 17 days. Checking, found that the DNS was set to "DNS only" and not "Proxied" in Cloudflare, so that may not be the reason for the auto-renewal to fail. Running wacs.exe, it mentioned that a scheduled task has a different path to the executable and had problems. Ah! I had moved the location of win-acme - that was the reason.
Ran wacs with the Edit option, choosing to edit "Everything", choosing the defaults for everything except the step of "remove existing scheduled task" for which I chose "Yes". Then it renewed and set up auto-renewal again.
Edit March 2023: Shifted to Cloudflare origin certificate since this problem repeated. https://hnsws.blogspot.com/2023/03/setting-up-cloudflare-origin.html