live streaming with stock laptop

Used the daily-use machine for live streaming on 23 November, with just its built-in soundcard. Reaper was running as something for me to monitor the waveform and also as an archive copy of the stream. Srv ran the mix through his Yamaha mixer from Chennai and used the Alesis compressor from the Studio to give me a compressed mix. My nominal input level for 0 dB for -10 dBV signal was as shown.

I rode the rec-level fader since Srv was monitoring before the compressor. Maybe next time a better choice of compressor settings can avoid my having to ride the faders. Anyway, output level was respectable, near -12 dB RMS, after the first few minutes when I was not riding the faders close enough and the compressor was not set on the output.

direct download to kindle

Playing around with the Kindle got for A - Latest Generation Wifi. Can download directly to the kindle by browsing gutenberg.org. But can't load to the kindle from email attachments. I think it's because the kindle's experimental browser does not support redirects and most webmail services have a redirect for attachments. Pointing the kindle to the direct link after deciphering the redirect's URL encoding, it loads the content with no issues. This works with the content emailed by yourkindlename@free.kindle.com too.

smtp timeout errors

After introducing the Cyberoam 50ia as the gateway at the studio, there were some teething troubles. When one of the two redundant links went down, the device would refuse incoming connections on the other link also. The management services would then have to be restarted from the console. Also, occasionally krishna had some smtp timeout issues even after the links were up. These were solved by restarting krishna.

qshape for checking postfix queue

One of the tools for checking the sending of our emails - qshape.
Some results from 5 days back when the problem was first detected:

sudo qshape -s incoming active deferred
shows

T 5 10 20 40 80 160 320 640 1280 1280+
TOTAL 33626 0 0 0 0 0 0 0 4 9645 23977
radiosai.org 32886 0 0 0 0 0 0 0 4 9645 23237
sssbpt.org 740 0 0 0 0 0 0 0 0 0 740

Out of these mails, most of them were yahoo.

sudo qshape incoming active deferred
T 5 10 20 40 80 160 320 640 1280 1280+
TOTAL 33623 0 0 0 0 0 0 0 4 9645 23974
yahoo.com 24602 0 0 0 0 0 0 0 3 6365 18234
yahoo.co.in 7809 0 0 0 0 0 0 0 1 2903 4905
airtelbroadband.in 154 0 0 0 0 0 0 0 0 30 124
yahoo.in 127 0 0 0 0 0 0 0 0 72 55
spml.co.in 82 0 0 0 0 0 0 0 0 20 62
lycos.com 67 0 0 0 0 0 0 0 0 13 54
....  snip  .....

bug - some files not imported into database

When there is a file with the same download file name (spaces replaced by underscores) being imported, the import script silently fails. The workaround would be to change the download filename of the old file in the database and then do the import.

importing files to downloads without playing them

1. Create playlists for the required files. Don't forget Signature tune. Choose a date in 1985 or something like that.
2. Enter suitable date in /home/sgh/scripts/playlist_import_hardcoded_date.sh
3. Then we can run playlist_import_hardcoded_date.sh to get the csv to the radiosai.org database
4. Then go to http://localhost/mkcsv/csv.php?date=01&month=01&year=1985 (or whatever date) to upload csv
6. Then go to radiosai.org and import the csv of the correct date.

bounce processing for yahoo and new ip address

Found a lot of delays from yahoo mail servers, deferred connections and so on. At first I thought it was just due to the new ip address - found that it was listed in Spamhaus, and sent in a request for removal from Spamhaus.
Contacted yahoo using their bulk mail contact form. Their reply said

After looking into the issue, it appears that your emails may have become de-prioritized due to a number of potential issues with your mailings. Please visit the following URL for information on improving list maintenance:
http://help.yahoo.com/l/us/yahoo/mail/postmaster/errors/postmaster-23.html
http://help.yahoo.com/l/us/yahoo/mail/postmaster/basics/basics-55.html
http://help.yahoo.com/l/us/yahoo/mail/postmaster/errors/421-ts03.html

The last point was about too many non-existent email addresses - and found lots of such errors in mail.log

Digging deeper, this is what I replied to yahoo:

Going through your checklist, and checking out our bounce processing, found that the bounces from yahoo were not being processed! The reason being that yahoo mail servers don't send the bounce messages, our mailer daemon has to send the bounce message, and this was not being sent to user noreply - user did not exist! This meant that more than 100 - 200 bad yahoo accounts have been sent mail from our server for the past several months! We're weeding out the bad ids - done some manually, rest will be processed by the bounce processing mechanism of phplist over the next week.

Actually there were around 400 bad mails of yahoo domains and a few from other domains. Unsubscribed the yahoo ones manually by digging through the bounces one by one using the gmail interface. The others will be caught by the bounce processing in due course.

For getting the bounces to the right place, I just had to create a noreply user, and put the following lines in a file /home/noreply/.procmailrc
MAILDIR=/var/spool/mail/noreply #The location of mail directory
:0
! our.bounces.email.id@gmail.com

After I sent the email to yahoo, they replied, saying in part,

Yahoo! Mail accepts a maximum of 20 messages per SMTP connection. We encourage you to cap the number of messages you send to Yahoo! Mail to fall within this per-connection limit.

My reply to that was:

We're running postfix > 2.3 which does not have a direct way to limit number of mails sent per connection.
http://www.postfix.org/CONNECTION_CACHE_README.html#limitations

Hopefully our server will reconnect OK with yahoo servers on dropping connections - our logs don't show any problems with that. We're still in the process of weeding out bad addresses. Once we finish that, hopefully, we won't irritate the yahoo MX servers as much!

In addition, I have now put
smtp_defer_if_no_mx_address_found = yes
in the /etc/postfix/main.cf - Default was to bounce mail if MX record not found, i.e. if DNS fails. Now changed to wait and retry for upto 5 days or whatever message age time is set.

download link from vimeo

Initially there was a requirement that we make the videos of MoL downloadable. P and I checked out various ways to do it from vimeo itself, since uploading duplicate copies to dl.radiosai.org would be not only tedious, but also tough on the server! Guided by this shell script, where he uses curl or wget to retrieve

http://www.vimeo.com/moogaloop/play/clip:${VIMEO_ID}/${REQUEST_SIGNATURE}/${REQUEST_SIGNATURE_EXPIRES}/?q=sd" > ${VIMEO_ID}.flv

where he gets REQUEST_SIGNATURE and REQUEST_SIGNATURE_EXPIRES from

VIDEO_XML=`${GET_CMD} http://www.vimeo.com/moogaloop/load/clip:${VIMEO_ID}`
REQUEST_SIGNATURE=`echo $VIDEO_XML  sed -e 's/^.*\([^<]*\)<.*$/\1/g'` 
REQUEST_SIGNATURE_EXPIRES=`echo $VIDEO_XML  sed -e 's/^.*\([^<]*\)<.*$/\1/g'`

P made a very elegant PHP5 script based on

$xml = simplexml_load_file("http://www.vimeo.com/moogaloop/load/clip:$id");
$request_signature = $xml->request_signature;
$request_signature_expires = $xml->request_signature_expires;
$downloadurl = "http://www.vimeo.com/moogaloop/play/clip:$id/$request_signature/$request_signature_expires/?q=sd";

Unfortunately our server did not have PHP5, and PHP4.x did not have the simplexml functions. So I modified it to use curl using the code example from this page and using the value_in() example from this page.

$xml = get_html("http://www.vimeo.com/moogaloop/load/clip:$id");
$request_signature = value_in("request_signature", $xml) ;
$request_signature_expires = value_in("request_signature_expires", $xml) ;
$downloadurl = "http://www.vimeo.com/moogaloop/play/clip:$id/$request_signature/$request_signature_expires/?q=sd";
echo "<p align = \"center\">";
echo "<a href=\"";
echo $downloadurl;
echo "\">Right-click here and choose 'Save Link As...' or 'Save Target As...'</a>";

Attempls to avoid the 'right-click instruction' did not work, since the vimeo server sends the video as mp4 without a Content-Disposition attachment header, and redirects don't conserve headers. Finally, of course, the script was not used due to non-technical reasons....