Thursday, June 18, 2026

rewrite of restic backup script and adding email alert

Found that our restic backup script had stopped running from Apr 20th - the log files were dated as last modified on that date. This could have been due to 
(a) an update overwriting the restic binary with an older version of ubuntu's restic via apt, which did not work, giving "backend not supported" errors for Azure Storage
(b) an additional issue that our Azure Storage account had become unavailable at some point of time due to a lapsed subscription 

So, created a new Storage account in an active subscription in the same resource group, which defaulted to the same Central India region for the storage account as we wanted. Used the same name as the earlier script's storage account name, and created a Blob container also with the same name as used by the script. 

Due to the point (a) above, this was not sufficient to make the script work, even for it to initialize the storage with lines like
restic -r azure:our-data-bk:/ourdata init
Fatal: create repository at azure:our-data-bk:/ourdata failed: invalid backend

As suggested by Gemini, the solution was to uninstall restic via apt, and to then download the latest restic and copy it to /usr/local/bin

wget https://github.com/restic/restic/releases/download/v0.18.1/restic_0.18.1_linux_amd64.bz2
bzip2 -d restic_0.18.1_linux_amd64.bz2
chmod +x restic_0.18.1_linux_amd64
sudo mv restic_0.18.1_linux_amd64 /usr/local/bin/restic

sudo apt remove restic

Then,
restic version
-bash: /usr/bin/restic: No such file or directory

We needed to tell bash to refresh the location - 
hash -r

Then the restic version showed the correct version, and the init also was successful. 

I wanted to get emails on future failures instead of the script failing silently, so Gemini helped with this script, which I have modified to change the passwords etc

#!/bin/bash
#This will run Restic backups from cron.
export RESTIC_PASSWORD=ourpw
export AZURE_ACCOUNT_NAME=ourname
#export AZURE_ACCOUNT_SAS="sv=2022-11-02&ss=bfqt&srt=c&sp=not_used_this_time%3D"
export AZURE_ACCOUNT_KEY="mVthisisthekey99999wPlEA=="
# not using rclone+gdrive due to slow,timeouts,rate-limits
#RCLONE_CONFIG=/home/user/.config/rclone/rclone.conf
#create new repo
# https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage
#restic -r azure:our-data-bk:/ourdata init
# take backups
/usr/local/bin/restic  -r azure:sssvv-data-bk:/1data  --verbose backup  /var/www/1_data_disk/1_data/filedir > /home/user/1resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/2data  --verbose backup  /var/www/1_data_disk/2_data/filedir > /home/user/2resticlog.txt 2>&1
/usr/local/bin/restic  -r azure:sssvv-data-bk:/3data  --verbose backup  /var/www1_data_disk/3_data/filedir > /home/user/3resticlog.txt 2>&1
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
    SUBJECT="SUCCESS: Weekly Restic Backup"
    MESSAGE="Your weekly restic backup completed successfully."
else
    SUBJECT="ALERT: Weekly Restic Backup FAILED"
    MESSAGE="WARNING: Your restic backup FAILED with exit code $EXIT_CODE. Please investigate immediately."
fi
mail -s "$SUBJECT" "my@email.org" << EOF
$MESSAGE

Here is the log output:
--------------------------------------------------
$(cat "/home/user/3resticlog.txt")
EOF



No comments:

Post a Comment