wiping a hard disk in Linux

using dd

https://how-to.fandom.com/wiki/How_to_wipe_a_hard_drive_clean_in_Linux#Wiping_specific_files

several hours?

https://superuser.com/questions/262056/how-long-to-zero-a-drive-with-dd

https://unix.stackexchange.com/questions/9432/is-there-a-way-to-determine-the-optimal-value-for-the-bs-parameter-to-dd

adding headers to s3 bucket urls

Exploring ways of migrating our downloads site to an AWS S3 bucket, we would preferably need to add the equivalent of
  ForceType application/octet-stream
 Header set Content-Disposition attachment

to prevent the files from just playing in the browser instead of downloading.

Looks like we could programmatically add the "content-disposition attachment" header to each file uploaded (to a folder) on an S3 bucket using a technique explained here,

https://stackoverflow.com/questions/19508318/how-to-set-content-disposition-headers-as-a-default-on-amazon-s3-bucket/51751708

redirecting specific php pages

I used the technique mentioned at this page and edited some of our php/html files to automatically redirect to our new domain with the same request URI, by adding a line
require("redirector.php");

with redirector.php having the following:

<?php

$domain = $_SERVER["SERVER_NAME"];

$requri = $_SERVER["REQUEST_URI"];

$newlocation = "location: https://our.new.domain" . $requri;

if ($domain == "www.olddomain.org" || $domain == "olddomain.org" )  { 

   Header( "HTTP/1.1 301 Moved Permanently" ); 

   header($newlocation); 

}

?>

Those php pages which have html content should have the above "require" line right at the top, before the html or head tags.

In the html file, put in a javascript redirect instead.

if (loc.indexOf('https://www.olddomain.org')==0){

    window.location.href = loc.replace('https://www.olddomain.org','https://our.new.domain);

}

if (loc.indexOf('https://olddomain.org')==0){

    window.location.href = loc.replace('https://olddomain.org','https://our.new.domain');

}

using wget to mirror a website

In order to download all the files on a website, first I tried WebHTTrack, but it was not limiting itself to downloading files on that domain even after going to the experts option and choosing stay on same domain. Maybe I should have tried the filters as in this forum post. Anyway, found a simpler method using wget, so just did

wget \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains our.domain \
     our.domain/link/to/index.html

         

Android Speech Services waiting for network notification

To get rid of the notification that Speech Services (formerly Text-to-Speech engine) is waiting for network, I knew that it would have something to do with downloading only on Wifi etc. But the specific location of the toggle for this component was not in Google Play Store app, but in settings as mentioned at this page - https://www.nextpit.com/how-to-fix-speech-services-waiting-for-network-connection

So, Google app - Profile icon - Settings - Voice - Offline-speech - Auto update tab - (choose update at any time instead of only on Wifi) - and change it back after the update completes :)

I did a device restart to get rid of the notification after the above change, since the language which was reported as waiting for network was not showing any "Update" link when I tap on it. 

car ownership transfer - do it yourself online IF Aadhaar is linked

A useful video about how to transfer car ownership in India on second sale, doing it ourselves online - 
https://www.youtube.com/watch?v=BeSaGyxCrrw

BUT - the procedure works properly only if both the parties have Aadhaar-linked mobile numbers, and if the car registration is also Aadhaar linked. And the documents have to be sent to the RTO office at the end. So, this process might be specific to Kerala RTO. And the website and process keep changing, so we would probably need to check again if trying after a few months.  

redirecting to new domain

Added permanent redirects to our new domain, like this - 

I have now put the following

RewriteEngine on
 RewriteCond %{SERVER_NAME} =olddomain.org
 RewriteRule ^ https://newdomain.org%{REQUEST_URI} [END,NE,R=permanent]

in /etc/apache2/sites-available/olddomain.org-le-ssl.conf

of our server and restarted apache.

so that when google returns the Sai Inspires link, the user gets redirected to the correct page. But this redirection seemed to break an application on another domain of ours. PB corrected this by the following:

Have created a softlink and it seems to be working.

Did the following steps

mv /path/to/archive/www/problemdomain  /path/to/archive/www/problemdomain_backup

ln -s /path/to/media/www/problemdomain /path/to/archive/www/problemdomain

  

Google Apps Script to list bounced emails

The following script lists all emails which bounce. So there would be multiple rows with the same bouncing email id. Adding another column to the Google Sheet with something like
=UNIQUE(C2:C)
(where C is the column of email ids, assuming first row contains column headers)
will get the list of unique bouncing emails.

From https://stackoverflow.com/questions/63248390/google-scripts-bounced-emails

function getBouncedEmails() {

// Create a sheet in your project "BouncedEmails" and add this code in the script.
var sheet = SpreadsheetApp.getActive().getSheetByName('BouncedEmails');

// Fetch all the bounced emails using a query
var query = "from:(mailer-daemon@google.com OR mailer-daemon@googlemail.com)";

//Get the most recent 500 bounced email messages
//Change the number according to your requirement
GmailApp.search(query, 0, 500).forEach(function(thread) {
    thread.getMessages().forEach(function(message) {
        if (message.getFrom().indexOf("mailer-daemon") !== -1) {
          //get the emailAddress from the Header
          var emailAddress = message.getHeader('X-Failed-Recipients');
          //add a filter if you would like to write certain messages only
          //if(thread.getFirstMessageSubject() == "YOUR SUBJECT"){
                    // Save the data in Google Spreadsheet
                    sheet.appendRow([
                        thread.getLastMessageDate(),
                        thread.getFirstMessageSubject(),
                        emailAddress]);
          //}
        }
    });
 });
}

subjective notes about Google Apps Script

Some thoughts after the last few months working with Google Apps Script - 

1. It's far easier to write standalone Google Apps Scripts (GAS) - from Google Drive as New - More - Google Apps Script or from an existing Google Doc or Sheet from Menu - Extensions - Apps Script rather than creating php scripts which call the Google APIs. There are multiple reasons, like 
   * better documentation with example code
   * excellent third-party examples 
   * all the details of setting up an appropriate project, granting scopes etc are simplified, especially using classes like DriveApp, DocumentApp and so on.  
   
   
2. One of the disadvantages of these standalone GAS is they usually have a three to five second (approx) response lag when fetching data from a google sheet or elsewhere, probably due to new server connection, sanitizing, security checks etc. So, AJAX calls are recommended for a better user experience if using the scripts inside iframes.  
   
   
3. Another disadvantage would be the long urls which are not on our domain, but that can be worked around by encapsulating inside an iframe on our site as above. For this, we should remember to set the Sandbox mode to IFRAME.
   
   
4. Script execution slows down dramatically if html is being generated in a loop. Especially when each loop execution has several API calls. For example, the following code snippet takes around 10 seconds to execute for 20 matches - 
   

   while (imgfiles.hasNext()){ count++; if (count > 20) { // to add paging code here. break; } imgfile=imgfiles.next(); //if(imgfile.getName().indexOf(title)!=-1){ //Logger.log("Found: "+imgfile.getName()); // https://stackoverflow.com/questions/489340/vertically-align-text-next-to-an-image imgfilename=imgfile.getName(); imgfileid=imgfile.getId(); drivefile=Drive.Files.get(imgfileid); // https://stackoverflow.com/questions/61272952/how-to-get-a-link-of-the-thumbnail-of-a-google-drive-file-using-google-apps-scri html+='<div>'+count+'. '; html+='<img style="vertical-align:middle" src="'; // changing the size of thumbnail linkstr = drivefile.thumbnailLink; linkstr = linkstr.substring(0, linkstr.length - 3)+'350'; html+=linkstr; //html+=imgfile.thumbnailLink; html+='" title="'; html+=imgfilename; html+='" alt="'; html+=imgfilename; html+='" width="150" >'; html+='<span style=""> &nbsp; <a href="'+drivefile.webContentLink+'" target="_blank"> <button >Download</button></a></span>'; html+='</div>'; html+='<br>Title: '; html+=imgfilename; html+='<br>Description: '; html+=imgfile.getDescription(); html+=' <hr>'; }

   A better way to implement would be to do all the processing in the background and display asynchronously. Link to best practices.
   
   
   

migrating emails from one google account to another, and storing archived data in Shared Drive

Copy-pasting from some correspondence about email migration and ease of access of data using a Shared Drive in Google Workspace - 

The process for moving all old emails from your old account to the new account is documented at the top half of the page at

https://support.google.com/mail/answer/21289?ctx=EU_GMAIL_IMPORT_Col1

The process for setting up email forwarding, so that all new emails go to the new account, is at

https://support.google.com/mail/answer/10957

Regarding exporting and importing of contacts - 

1. If no contacts have been manually saved, then just importing the emails to the new account will probably enable the automatic pop-up of the email addresses

2. In case that does not happen, or if you wish to make the contacts visible immediately, you can go to contacts.google.com with the old login - the contacts which were not manually saved would be visible under "Other Contacts" on the left-hand side. Export has an option for "Selected contacts", so if you select the "Other Contacts", you can export them.

 

Yes, emails can be exported to Gmail, to the new account. 

But my understanding is that you have a lot more than 30 GB of emails (with attachments) which you wish to archive? In that case, once again you will have the mailbox becoming full. 

 

What you may want to do would be to drag and drop the required attachments to the Shared Drive which I have created for you, which you can access from 

Google Drive - Shared Drives (on the left hand side)

Mailstore seems to have the option to export to Directory (file system)

https://help.mailstore.com/en/home/Exporting_Email

After that export, you could drag and drop that directory (after checking if the attachments are also there) to the Shared Drive.

 

 

Moodle Front-page behaviour with Moove theme

We had a query about the site home page, also called front page in Moodle, for one of our installations which ran the Moove theme. If a website visitor does not log in, but instead clicks on the sample course, they do not see all the content in the site home page again if they click on Home or on the website Logo. The team was looking for a workaround, and my reply was something like the following:

I believe the "Guest user not seeing home page" behaviour is by design.

1. When a 'not-logged-on' user goes to the home page (index.php) - they are shown the entire "Home" page which includes the "Sponsors" and "Marketing" blocks.

2. When clicking on "Sample course", the user is then logged in as guest. When logged in, clicking on "home" or the site logo on top takes the user to the "Site home" page, which is the same index.php, but with the About us, FAQ etc shown in the "Main Menu" which may be collapsed on the Right-Hand-Side, and the "Marketing" blocks not visible. 

3. There is no "Logout" link directly available for Guest Login, except a "login" link to the login page (at login/index.php) - the only way the guest login will be logged out without logging as another user would be to
(a) close the session
(b) clear the cookies.

4. Moodle does offer an option to customize the front page -
https://www.inmotionhosting.com/support/edu/moodle/customize-front-page/

using "Add a block" and then choosing "HTML block" by going to the home page logged in as Administrator.

But then the look and feel which Moove's  "Sponsors" and "Marketing" blocks would need to be replicated, I suppose. 

I then made a test version of such an HTML block by copy-pasting from "view-source" for a non-logged-in user. Found that the image sizes in the "Marketing" blocks would need to be tweaked manually, but the rest worked reasonably well. 

Apache KeepAlive and surge of traffic

Came to this post
https://www.kalzumeus.com/2010/06/19/running-apache-on-a-memory-constrained-vps/

via Falsehoods programmers believe about names via slashdot.

So, KeepAlive will kill your Apache if there is a sudden traffic surge and you are RAM constrained. Good to know. Probably that's what caused our issues with the reverse-proxied streaming server. 

problem with audio recording and or upload in Android app

Excerpts from a conversation about an Android app under development and testing - 

You had reported that 2 out of the 20 devices tested had issues with the SainIn app audio recording + uploading.

We verified that the permissions for the app were correctly set.

We wanted to check if mp3 encoder is installed, maybe you could try installing https://play.google.com/store/apps/details?id=net.mp3.codec.pack&hl=en&gl=US which seems to have an option for "broken codec detection." 

Or maybe install this app -
https://play.google.com/store/apps/details?id=com.github.khangnt.mcp
which has mp3 encoder (for recording), while the other link above is only mp3 decoder (for playback).

But after installing this encoder app also, our app did not record.

Since one of the devices was a Moto E, which is known for not having mp3 codecs from the factory, I thought of looking at the Android spec to see which codecs can be reliably used -
https://developer.android.com/guide/topics/media/media-formats

Apparently AAC LC codec Encoder is available. 

There is a simple recorder app at
https://github.com/TannerGabriel/android-sound-recorder

which I built and sent to you, and which worked. 

There, the AAC encoder is being called, and output format is mp3. Relevant code seems to be at this location.

Google Workspace - migrating emails from one account to another in a different domain

Series of steps needed, after the new account is created by the admin:

For this process to work using POP3, first you have to enable pop3 on the old account - 

https://support.google.com/mail/answer/7104828?hl=en

Edit: Migrating using IMAP is much smoother, but probably needs admin access to the destination Google Workspace domain. My later post about moving an entire domain covers this method of migration, using Google Workspace Data Migration Service.

The process for moving all old emails from your old account to the new account is documented at the top half of the page at

https://support.google.com/mail/answer/21289?ctx=EU_GMAIL_IMPORT_Col1

with special attention needed for the "Server denied POP3 access" part, because with pop.gmail.com, we would need to go through all the steps at

https://support.google.com/mail/thread/22276276/pop3-gmail-accounts?hl=en

(At the end, you get the message "Your messages are being imported ... may take several hours ... up to two days ... You can close this wind

Then, the process for setting up email forwarding, so that all new emails go to the new account, is at

https://support.google.com/mail/answer/10957

Regarding exporting and importing of contacts - 

1. If no contacts have been manually saved, then just importing the emails to the new account will probably enable the automatic pop-up of the email addresses

2. In case that does not happen, or if you wish to make the contacts visible immediately, you can go to contacts.google.com with the old login - the contacts which were not manually saved would be visible under "Other Contacts" on the left-hand side. Export has an option for "Selected contacts", so if you select the "Other Contacts", you can export them.

multi-track audio with reaper

While reviewing old notes, found this - 
14 Jan 2020 - 1. Reaper does not export multilang multi wav properly - other tracks are muted. Maybe some extra config needed?
2. Need to start audio after 2 seconds, else clipped on playback.

This reddit thread led to 

• https://www.youtube.com/watch?v=Gm2bw7OUP_w
• https://forum.cockos.com/showthread.php?t=166073

By clicking on the IO in the Master channel, we have to set the Master to 6 channel instead of the default Stereo. Similarly, have to do that for each channel's track. 

Google Workspace - content compliance rules for GMail

A small gotcha - the content compliance rules for Gmail only display when the appropriate organizational unit is selected. Or else the content compliance rules are not visible (except those which are active for all organizational units).

Currently these settings come under Apps - Google Workspace - Settings for GMail - Compliance.