Tuesday, December 24, 2024

notes on backing up Azure blob storage folder to local external hard drive

Copy-pasting from an email exchange - 

As the message says, a few thousand files did not download properly, there were errors. Most probably due to bad internet connection. Would need to retry again and again until all are downloaded. Or, use a better internet connection - for eg, I use home BSNL fiber (100 Mbps) when I need to download large files. Another option is Airtel 5G, for one-time downloads. 

When I downloaded on BSNL Fiber using Azure explorer, the download went at around 50 Mbps, 111 GB in 17605 sec or just below 5 hours.

Copying from one hard disk to another showed another possible reason for the previous download to fail - the external hard disk was corrupted, leading to file copy failure - repaired the file system and the file copy worked OK.

If needed again, can explore rclone on the RPi.

No comments:

Post a Comment