changes needed for migrating show creation from Windows to Linux

As noted in an earlier post, 

https://hnsws.blogspot.com/2020/01/warping-on-linux-filter-on-avisynth-and.html

avisynth works on Linux under Wine, with certain changes.

• need to run Virtualdub with
  cd ~/windows/VirtualDub
  wine VirtualDub
• need to add the line 
  Loadplugin("/home/myusername/.wine/drive_c/Program Files/AviSynth 2.5/plugins/ffms2.dll")
• need to use FFvideosource("in.mp4") instead of DirectShowSource()

The wav to 5.1 AC3 is done well by ffmpeg,
https://hnsws.blogspot.com/2020/01/6-mono-waves-to-51-ac3-using-ffmpeg.html

streaming our own youtube videos as a live stream on youtube

Implemented as a bash shell script using ffmpeg and the latest youtube-dl fork yt-dlp, at https://github.com/hn-88/live-stream-from-youtube-to-youtube 

References are mentioned in the readme. Usage is in the comments.

trying out the free email check api

Mailboxlayer.com offers a free tier for their email verification api, 100 requests free per month - I signed up for free, and could easily use their api - via apilayer.com.

The api (from https://blog.apilayer.com/the-anatomy-of-a-great-email-address-validation-api/) in the format

http://apilayer.net/api/check?access_key=4aMyAccessKey6f8ca4&email=thisIsWhatWe@WantTo.Check&smtp=1&format=1

The smtp_check value (in the returned data) being false would indicate email not existing. This is probably the older version of the mailboxlayer api, since the current api has a syntax like

curl --location --request GET 'https://api.apilayer.com/email_verification/customercare@apilayer.com' \

--header 'apikey: YOUR API KEY HERE'

turning the alarm off on HTC-1 clock humidity temperature meter

As seen in this video, in order to set the alarm, short-press Mode, then long press Mode till minutes start blinking. In order to turn alarm on or off, press mode again, set the hours, press mode again, and now when in Alarm setting mode without the numbers blinking, press the Adj button to toggle through Alarm on, (tone every 15 minutes I think) and Alarm off. If the unit is just left alone for a minute, the current time display returns. Or we can short-press Mode again to come out of Alarm set mode. 

So, in short, if we just want to toggle Alarm on or off, just short press Mode and then short press Adj repeatedly to toggle through the Alarm on or off modes.  

mass emailing with Google Apps scripts and validating email lists

One of the ways in which we can send "Mail merge" equivalent customized emails to a large number of users would be using the sample script at
https://developers.google.com/apps-script/samples/automations/employee-certificate

- this seems to be a much better solution than GMass (free option) which I had earlier posted about, but requires some coding. 

This works quite well if all the emails are pre-verified. Our non-profit accounts have ~1500 emails per day as reported by

var emailQuotaRemaining = MailApp.getRemainingDailyQuota();

Logger.log("Remaining email quota: " + emailQuotaRemaining);

 
But the important thing is that the list of emails must be accurate. If multiple emails bounce, the whole process will get derailed, and the user sending the emails will get suspended. We should probably also go slowly so that rate limiting does not kick in. 

One of the good ways to ensure valid emails might be if all these emails in the list are obtained using the users filling up some google form which requires google sign-in. 

A youtube video for validating email lists uses SMTP check being done with api from

https://mailboxlayer.com/product

An alternative might be https://www.myfreshtools.com/misc/bulk-email-validator-tool

More ways to do smtp check - 

https://stackoverflow.com/questions/565504/how-to-check-if-an-email-address-exists-without-sending-an-email

We could also add the email ids to a google group - that would also validate their email to some extent, with at least hotmail, yahoo and gmail ids being caught if invalid. 

Edit: Some interesting reading about spam filters and "warming up" - 

https://www.gmass.co/blog/free-email-warm-up-tool/

webpage with links to ebook using RSS and Calibre

Calibre has the feature of "news feed" which will fetch the links listed in an rss feed and create an ebook out of them. Though "custom recipes" can be written to scrape pages and get links from there, I tried to make things easier for me by using an automated tool to create RSS from HTML, and then use calibre's "add new recipe" to "create a recipe from scratch" using just the RSS feed. The Hindu's Young World archives did not have an RSS feed, but I could create one quite easily with feed43 ("Feed for free") using the simple item search pattern

<h3><a href="{%}"{*}class="">{%}</a></h3>

with

Item link template = {%1} and 

Item Title template = {%2}

feed43's free feed is limited to 20 items, so for all the older links, I would need something else. Found mkfeed, which uses the same syntax as feed43. So I just needed to install mkfeed and slightly tweak the provided example to create an RSS file locally - 

URL="https://www.thehindu.com/topic/The_Hindu_Young_World/"

wget -q -O - "$URL" | mkfeed \

    --pattern-item '<h3><a href="{%}"{*}class="">{%}</a>' \

    --feed-title 'YW Archive1' \

    --feed-link "$URL" \

    --feed-desc 'Older Young World articles' \

    --item-title '{%2}' \

    --item-link '{%1}' \

    --item-desc '{%2}' > yw1.rss

Calibre didn't seem to be able to fetch local files as RSS feeds when I gave the local path as /local/path/to/yw1.rss as the feed url. So, ran a simple webserver using python as described here, but changing the port to the less privileged 8000, with

import os

from http.server import HTTPServer, CGIHTTPRequestHandler

# Make sure the server is created at current directory

os.chdir('.')

# Create server object listening the port 8000

server_object = HTTPServer(server_address=('', 8000), RequestHandlerClass=CGIHTTPRequestHandler)

# Start the web server

server_object.serve_forever()

Then it was just a matter of pointing a new recipe in calibre to http://localhost:8000/yw1.rss and calibre did the rest. This might be an easier set of steps to do compared with my earlier manual method using HtmlAsText.

phone photos as thumbnails in Nemo

In order to manage the photos on my Android phone, it is convenient to connect it via USB and browse the filesystem via the PC (Linux Mint) File Manager, Nemo. If I want to see the image files with their thumbnails as icons, have to change Edit - Preferences - Preview - Show thumbnails from Local files only to Yes. Then, it will show thumbnails for previewable files even for remote filesystems like the Android filesystem. Also have to change the "Only for files smaller than" setting, for which the default is 1 MB. 

Edit - and then increase the size of the icons using the slider on the bottom right of Nemo's status bar, like below - 

google apps script for adding to a google group

Since we have a non-profit edition of Google Workspace, we can use google groups with group email id like email@ourdomain.tld for a group created using the admin console, and users can be added to such groups using the API. My trial scripts have been added to this github repo, Google-groups-GAS-tests. Some points to note - 

• As seen in the medium post I used as reference, we have to give a sleep statement after calling group.hasUser() to avoid errors.
• We can't use the add or remove functions on @googlegroups.com groups, but listing users etc works.
• The AdminDirectory service in Google Apps Script is lacking in documentation, as are the other services, with AdminDirectory.Members.remove() being an example of an insufficiently documented function - in the REST API, the function is called delete. Most people suggest using the autocomplete option in the script editor to find documentation about the advanced services!
• When calling the deployed GA script as a web app, the Logger.log logs don't appear in the Executions tab by default - even removing the tick for "Log uncaught exceptions" did not help - looks like we need to associate the project with a GCP project in order for that to work. Or, apparently, console.log is the preferred method now. 
• We should be aware of the limits -
  https://support.google.com/a/answer/6099642#zippy=%2Cmessage-and-posting-limits says Total external recipients** per group = 50,000 per day So, if we need a cushion to send more than one message per day, we would need to keep the number of users below 25k.  

google apps script to turn blog posts into an ebook - no oauth required

It turns out that all the complicated steps I went through in the previous post were not necessary. Blogs on blogger provide a feed url which returns xml or json as you prefer, with no need for authentication for public blogs. My code is at https://github.com/hn-88/bloggerToEbook and the feed url, for reference, is like 

https://domainnameofblog.tld/feeds/posts/default/-/relevant-label?max-results=100&alt=json

If we want to narrow down results via posted date,

https://developers.google.com/blogger/docs/2.0/reference

shows us that we need to add the following to the url above - 

&updated-min=2008-03-16T00:00:00&updated-max=2008-03-24T23:59:59&orderby=updated

After downloading the Google Doc as an EPUB, I did a couple of tweaks to make it read better on the Kindle when converting to MOBI with Calibre. Turning the green text to black, by ticking the 'Color' box to ignore,

and also auto-generating Table of Contents,

I had also increased the number of links to add to Table of Contents from 50 to 500 in the screen above, since I was making ebooks with 100 or 150 posts each. 

Edit: Bloxp.com is one of the high ranked google results for turning a blog into an ebook. But right now it doesn't seem to be working - tried both with Blogger feeds as well as rss feeds from https://www.thehindu.com/rssfeeds/

Edit 2: Another way to turn blog posts into an ebook, mentioned at

https://ravisiyer.blogspot.com/p/consolidated-blog-content-document.html

is

 http://blog2book.pothi.com/app/

This seems to be dead now. archive.org shows that it redirects to pothi.com now. 

google apps script to turn blog posts into an ebook - calling blogger api

Google Docs have an EPUB export option. So, if we get the html content of the blog into a google doc, we can export as an ebook. Some blogger blogs have good content, but have some sort of html / CSS / char encoding issues so that merely doing

var response = UrlFetchApp.fetch(urls[i], { muteHttpExceptions: true });

 var ablob = response.getBlob();

      var AssetGDocId = Drive.Files.insert(

        { title: ' temp' + i + '.html', 

        mimeType: MimeType.GOOGLE_DOCS },

        ablob

      ).id;

results in a google doc with a single character per line!

So, going the blogger api route. But there are certain complications.

Blogger is not one of the services which are directly available via the "Add a service" dialog box,

When following the example at

https://www.labnol.org/code/20068-blogger-api-with-google-apps-script/

needed to make a couple of changes as mentioned at

https://github.com/googleworkspace/apps-script-samples/issues/121

making the headers variable

Authorization: "Bearer " + ScriptApp.getOAuthToken(),

and adding 

"oauthScopes": [

    "https://www.googleapis.com/auth/drive",

    "https://www.googleapis.com/auth/blogger",

    "https://www.googleapis.com/auth/script.external_request"

  ],

to the manifest. To add to the manifest, we have to make it visible in the editor via the settings tab,

https://developers.google.com/apps-script/concepts/manifests

Further, the default apps script project does not have the blogger api automatically enabled as for other advanced services - we have to create a project and add the blogger api (and drive api in this case). We can ignore creating credentials - the app script will automatically create the oauth credentials. 

Once we create a new project at console.cloud.google.com, we have to add the required apis using the APIs and services tab on the left-hand side - Blogger api and Drive api in this case. And then associate the google apps script with that project under the script settings, Change project. The project number can be found from the Home -> Dashboard page, in the project info card of console.cloud.google.com.

A caution here - if creating credentials or editing the consent screen - sometimes there are errors creating credentials, and one of the triggers seems to be the use of multiple logins. If we are logged in to the cloud console by appending &authuser=2 (or something like that) to the url, like 

https://console.cloud.google.com/?cloudshell=true&pli=1&authuser=2

then credentials creation can run into all sorts of unexplained errors. "There was an error creating credentials...."

It would be safer to log in with only one account, perhaps by using incognito mode. Or maybe this firebase trick or other issues in this post can help in some cases.

After creating the consent screen, we have to add the scopes needed - again, drive and blogger apis in this case. And also add our own email id as a test user. 

Finally, after getting the api calls to work, I need to find out how to get all the posts without timing out. One way might be to use the search call instead of the list call and get "all posts labelled with 'suitable label' in September 2013" or something like that. That would probably be covered in a separate post here.

Edit: Getting all the posts with a particular label or a particular date range is a good way to narrow down the number of posts, and that is how I implemented. But it turned out that all this work of adding the blogger api etc was not required after all - the blogger atom feed api doesn't need any authentication for public blogs.