Tuesday, July 07, 2026

reminder to take a break - Mac version

I wanted to recreate the earlier "reminder to take a break" cron job script on MacOS. Asked Claude about it, and it suggested using launchd instead of cron, since cron jobs run without access to the display by default. After several rounds of trial and error, here is the method that worked.

nano ~/Library/LaunchAgents/com.yourname.walkreminder.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.yourname.walkreminder</string>

    <key>ProgramArguments</key>
    <array>
        <string>/bin/bash</string>
        <string>-c</string>
        <string>osascript -e 'display notification "Time to take a walk" with title "Reminder" sound name "Ping"'; afplay /Users/yourusername/Sounds/Walk.mp3</string>
    </array>

    <key>StartCalendarInterval</key>
    <array>
        <dict>
            <key>Hour</key>
            <integer>14</integer>
            <key>Minute</key>
            <integer>0</integer>
        </dict>
    </array>

    <key>StandardOutPath</key>
    <string>/tmp/walkreminder.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/walkreminder.err</string>
</dict>
</plist>

In the above, the schedule is to run once a day at 14:00 hours. For running every half an hour from the time the user was logged on, we can use 

<key>StartInterval</key>
  <integer>1800</integer>

instead of <key>StartCalendarInterval</key>

Or, in my case, since I wanted every half an hour aligned to 0:00 and 0:30 exactly, 

 <key>StartCalendarInterval</key>
    <array>
        <dict>
            <key>Minute</key>
            <integer>0</integer>
        </dict> 
        <dict>
            <key>Minute</key>
            <integer>30</integer>
        </dict>
    </array>

We had to use a Sounds directory which we had created, since afplay could not access the Downloads directory when run from launchd.

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.yourname.walkreminder.plist

and test with

launchctl kickstart -k gui/$(id -u)/com.yourname.walkreminder

Troubleshooting can be done with
plutil -lint ~/Library/LaunchAgents/com.yourname.walkreminder.plist
launchctl list | grep walkreminder
cat /tmp/walkreminder.err


Friday, July 03, 2026

incident and response - malicious bots

One of our servers, which hosted a dot net backend as well as some wordpress sites, showed some outages two days in a row, restored by restarting. Feeding the apache access logs to Gemini via aistudio.google.com, Gemini said,

The logs clearly show an Apache HTTP
Server running PHP and hosting a WordPress application.

The logs indicate that your server was hit by an aggressive automated attack
(likely a botnet) starting around 07:33:35, which appears to have either
succeeded in exploiting a vulnerability or overloaded the server, causing it to
crash shortly after.

(Server had run out of memory). 

Gemini suggested the following - 

  1. Hide and Protect Your Origin IP - which we may not do
  2. Patch and Update the Server Software - which we're doing
  3. Implement a Web Application Firewall (WAF) - which is in place
  4. Clean Up WordPress Configuration - "Open your wp-config.php file, locate line 63, and remove the    duplicate definition of WP_AUTO_UPDATE_CORE. While this didn't cause the crash, it eats up server I/O and makes reading logs difficult."
  5. Restrict PHP Execution in Uploads Directories
  6. Implement Intrusion Prevention (Fail2Ban)

Then, feeding the access logs of the dot net api to Gemini 
and as suggested by Gemini,
journalctl -u ourapi.service --since "2026-06-28 07:15:00" --until "2026-06-28 07:45:00"

Gemini gave some recommendations like returning 0 instead of a 500 error for no data found -
"As we saw in the access logs, when the mobile app sees
    this 500 error, its poorly designed error-handling logic says: "Something
    went wrong! Retry the entire sync process!" It then proceeds to download 4MB
    of lesson data, hits the notification endpoint again, gets another 500
    error, and repeats the cycle every 15 seconds until your server runs out of
    memory and crashes."
- this may or may not be actually what is happening here, since Gemini just saw the logs and does not have access to the code.

In any case, I've enabled bot protection on Cloudflare for this domain, and also installed and configured Fail2ban.

On Cloudflare, 
ourdomain.org > Security > Security rules > Custom rules

Block .env scans
URI Path equals .env
Block

 also added Cloudflare's default rate limiting rule,

Leaked credential check [Template]
Password Leaked equals true
Block

also enabled Cloudflare's AI blocking tools - AI Labyrinth enabled, and Block AI training bots on all pages.

Then, installed and set up Fail2ban on the server, monitoring the apache web server and sshd logs. It will temporarily block ip addresses which fail authentication repeatedly, or which repeatedly request non-existent files like malicious bots.

Gemini via aistudio.google.com gave step-by-step instructions for installation and setup of Fail2ban. 

sudo apt install fail2ban -y
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

(we can optionally edit these,
bantime  = 1h
findtime = 10m
maxretry = 5
)

Enable the Apache jails with
sudo nano /etc/fail2ban/jail.local
[apache-auth]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

In case the logpath is custom - fail2ban looks by default for
Error logs: /var/log/apache2/*error.log
Access logs: /var/log/apache2/*access.log

In the case of one of our servers, we needed to change this to
logpath  = %(apache_error_log)s
           /var/www/mysite/logs/custom-error*.log

More Apache jails to enable - basically need to add the enabled = true line - 

[apache-badbots]
enabled  = true
port     = http,https
logpath  = %(apache_access_log)s
bantime  = 48h
maxretry = 1

[apache-noscript]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-overflows]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-nohome]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

[apache-botsearch]
enabled  = true
port     = http,https
logpath  = %(apache_error_log)s

Then we can test

sudo systemctl start fail2ban
sudo systemctl enable fail2ban
sudo systemctl status fail2ban

and check the enabled jails with

sudo fail2ban-client status

On one server, we had to resolve
ERROR   Failed during configuration: Have not found any log file
for sshd jail

For that, we had to change the block to
[sshd]
enabled = true
port    = ssh
backend = systemd

since that server was using systemd, and traditional text log files like /var/log/auth.log (which Fail2Ban looks for by default for SSH) are no longer created. After the change, after restarting the service, we can check that jail alone with

sudo fail2ban-client status sshd

Then, for using a different port rather than port 22 for sshd - 

[sshd]
enabled = true
port    = 2022
backend = systemd
(if port 2022 is being used) 
and so on, because that is the port which fail2ban would ban using the machine's firewall, iptables or nftables as the case may be.

To see the rules, we can use
sudo nft list ruleset # for nftables, Ubuntu 24.04
sudo iptables -nL # for iptables, earlier Ubuntu etc



Tuesday, June 23, 2026

SIR online form submission - Election Commission of India

Currently, a "Special Intensive Revision" or SIR is going on for the elector rolls in many parts of India. My colleague told me about the process going on in the local administration's office, and told me that I could submit the form online also.

An internet search revealed

Luckily, my mobile phone number was already available in the system, so when I clicked on sign-up, it indicated that my number is already registered. Then tried log in - an OTP was sent to my phone, and I could log in. 

The process which I followed was
  • Using the search function, found my name in the 2002 electoral rolls - it was on page 2 or 3, I skimmed through the results based on polling station name (we need to be able to read Telugu, the local language, for doing this.) I saved the info listed there - Assembly constituency, polling station and  
  • Next, went to the home page and clicked on "Fill Enumeration form"
  • We have to go through several OTPs, confirm our details, ensure that the name displayed is the same name as on Aadhaar - otherwise we need to do the form submission manually, details of the BLO (Booth Level Officer) are displayed.
  • Then the form opens up, where we need to fill up details and a recent photograph. The initial photographs I uploaded were not recognized "no face was recognized in this photograph" or something like that, after a wait of a minute or two. Thinking that this might be due to spectacles in the picture, clicked a new photo of myself without spectacles, and tried the upload.
  • Yesterday at around 2.30 pm, the form came up for submission after that, though the photo was not visible in the preview. But when I clicked on submit, the "Aadhaar OTP" method of verification did not succeed, it said wrong OTP every time.
  • This morning at around 8.50 am, went through the same process, this time the "Aadhaar OTP" method of verification was successful - again had to wait for a few minutes after uploading the photo in order to get the submit form enabled - and I got a downloadable receipt of the form submitted, with the new polling station number, constituency name and so on.
  • Found that I could also download my voter-id card (EPIC - Election Photo Identity Card) - from https://voters.eci.gov.in/ 

Thursday, June 18, 2026

rewrite of restic backup script and adding email alert

Found that our restic backup script had stopped running from Apr 20th - the log files were dated as last modified on that date. This could have been due to 
(a) an update overwriting the restic binary with an older version of ubuntu's restic via apt, which did not work, giving "backend not supported" errors for Azure Storage
(b) an additional issue that our Azure Storage account had become unavailable at some point of time due to a lapsed subscription 

So, created a new Storage account in an active subscription in the same resource group, which defaulted to the same Central India region for the storage account as we wanted. Used the same name as the earlier script's storage account name, and created a Blob container also with the same name as used by the script. 

Due to the point (a) above, this was not sufficient to make the script work, even for it to initialize the storage with lines like
restic -r azure:our-data-bk:/ourdata init
Fatal: create repository at azure:our-data-bk:/ourdata failed: invalid backend

As suggested by Gemini, the solution was to uninstall restic via apt, and to then download the latest restic and copy it to /usr/local/bin

wget https://github.com/restic/restic/releases/download/v0.18.1/restic_0.18.1_linux_amd64.bz2
bzip2 -d restic_0.18.1_linux_amd64.bz2
chmod +x restic_0.18.1_linux_amd64
sudo mv restic_0.18.1_linux_amd64 /usr/local/bin/restic

sudo apt remove restic

Then,
restic version
-bash: /usr/bin/restic: No such file or directory

We needed to tell bash to refresh the location - 
hash -r

Then the restic version showed the correct version, and the init also was successful. 

I wanted to get emails on future failures instead of the script failing silently, so Gemini helped with this script, which I have modified to change the passwords etc

#!/bin/bash
#This will run Restic backups from cron.
export RESTIC_PASSWORD=ourpw
export AZURE_ACCOUNT_NAME=ourname
#export AZURE_ACCOUNT_SAS="sv=2022-11-02&ss=bfqt&srt=c&sp=not_used_this_time%3D"
export AZURE_ACCOUNT_KEY="mVthisisthekey99999wPlEA=="
# not using rclone+gdrive due to slow,timeouts,rate-limits
#RCLONE_CONFIG=/home/user/.config/rclone/rclone.conf
#create new repo
# https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage
#restic -r azure:our-data-bk:/ourdata init
# take backups
/usr/local/bin/restic  -r azure:sssvv-data-bk:/1data  --verbose backup  /var/www/1_data_disk/1_data/filedir > /home/user/1resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/2data  --verbose backup  /var/www/1_data_disk/2_data/filedir > /home/user/2resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/3data  --verbose backup  /var/www1_data_disk/3_data/filedir > /home/user/3resticlog.txt 2>&1
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
    SUBJECT="SUCCESS: Weekly Restic Backup"
    MESSAGE="Your weekly restic backup completed successfully."
else
    SUBJECT="ALERT: Weekly Restic Backup FAILED"
    MESSAGE="WARNING: Your restic backup FAILED with exit code $EXIT_CODE. Please investigate immediately."
fi
mail -s "$SUBJECT" "my@email.org" << EOF
$MESSAGE

Here is the log output:
--------------------------------------------------
$(cat "/home/user/3resticlog.txt")
EOF



Wednesday, June 17, 2026

Updating expiring Azure Linux Virtual Machine Secure Boot 2011 certificates

Microsoft Azure's email notification asked us to update the secure boot certificates before the end of the month, and pointed us to verification, and if necessary updating, steps. The "vendor recommended" documentation for Ubuntu support was a bit contradictory - saying that rollout had been paused - so took the help of ChatGPT and Gemini for completing the process. First took up a non-critical VM, completed that, and then went on to the others.

sudo snap install fwupd
sudo fwupdmgr refresh
sudo fwupdmgr update
#(say yes, yes, and yes to reboot)

Gemini reassured that the devices listed with "no updates" are not a concern, we should only check whether the mokutil tests below work OK.

As per the verification link above, 
Tested with
mokutil --db | grep "2023"
            Not Before: Jun 13 19:21:47 2023 GMT
        Subject: C=US, O=Microsoft Corporation, CN=Microsoft UEFI CA 2023
mokutil --kek | grep "2023"
            Not Before: Mar  2 20:21:35 2023 GMT
        Subject: C=US, O=Microsoft Corporation, CN=Microsoft Corporation KEK 2K CA 2023

Removed the fwupd snap, and also removed snap itself to prevent bloat
sudo snap remove fwupd
(and remove snap itself on ELS)
snap list
(if nothing other than core, core20, lxd, or snapd, can remove)

sudo systemctl disable --now snapd.service snapd.socket
sudo apt-get purge -y snapd
sudo rm -rf /snap /var/snap /var/lib/snapd /var/cache/snapd /usr/lib/snapd

Found that the L VM was already up-to-date since it was a newer VM, created in 2025.

SDev2 had to be updated in the same manner as for the HAPROXY VM above.

SSS web server also had to be updated in the same manner.

On the AWS VM, I see

mokutil --sb-state
EFI variables are not supported on this system

Gemini says,

You do not need to do anything for this AWS VM. You are completely in the clear.
Seeing EFI variables are not supported on this system means that this specific EC2 instance is not using UEFI Secure Boot at all. In fact, it is likely booting using Legacy BIOS rather than UEFI.


With that, all the VMs seem to be accounted for.

Tuesday, June 16, 2026

Samsung phone Gboard voice typing icon vanished

At some point of time, after some upgrades / updates, the voice-typing mic icon vanished, for the Gboard keyboard on my Samsung M34 5G phone. 

There were lots of contradictory (and perhaps out-dated) info online on how to re-enable voice-typing, but what worked for me was as follows. Gboard was already set as the default keyboard, and not Samsung keyboard in the settings. 

When Gboard was visible, the path to set this was,
the icon to show more items - 
then the Gboard settings icon,

 followed by voice-typing

and finally slide to enable "Use voice typing" as below.

Then the mic icon at the top right corner of Gboard becomes visible.