CSV list for phplist mailing list

H wanted a new mailing list, for which he supplied email addresses in an excel sheet. The easiest way to import these into phplist, since he did not want any customization, was as a simple list of emails, one per line - phplist can import such a list with minimum of fuss. Unfortunately, even that took a while - the list had many typos, so phplist was barfing it up. After H did cleanup using some tool, tried the 18,000 email list at once, phplist did not respond. Then cut it up into smaller pieces - upto 6000 emails per file was imported without any problem.

More CDNs

A follow-up to my CDN roundup.

Akamai offered minimum commit of Rs. 1.5 lakhs per month.
$2000 for 2000 GB, around $1 per GB till 10 TB.

Internap offered a free unlimited streaming account till Dec 30th, but pricing was minimum commit $1000 with traffic charged at $2 per GB. Also, using their test account, found:

• the sample flash player provided by internap was crashing in Safari
• the sample flash player also did not allow the hiding of controls - controls were always visible onscreen
• we were unsuccessful in getting jwplayer to work with the internap stream, even after the suggested setting of rtmp.subscribe flashvar to true.

converting stereo wav files to mono

A wanted some advice on converting interview audio files recorded in stereo to mono.

In Sound Forge: To mute only one channel, move the cursor to the top quarter (or bottom quarter) of the screen. The cursor will get a small subscript of L or R (for left or right channel). Select the area you want to mute - only the L (or R) side will get selected. Now if you choose Process -> Mute, only that channel will get muted.

After going through this process of muting out unnecessary parts in L and R channels separately, you can bring the whole wav back to mono using Process -> Channel Convertor. You should use the built-in preset
Stereo to Mono use both channels 50% (no faders)
in the Channel convertor drop-down list for this.

shutting down a Windows server remotely

H & N wanted a method of shutting down their server from the client machine at the counter. Client running Windows XP, server running Win2003 server. From Microsoft documentation, an example batch file (contents shown below), which can be put on the desktop, and double-clicked to shutdown the server. You will need to change the servername in the batchfile for this to work, and of course rename it as something.bat .
type "Are you sure you want to shut down the server? Press any key to continue"

Pause
shutdown -m \\servername -t 10 -c "Shutting down server in 10 seconds"


To start up the server, I mentioned Wake up On Lan to them, but their BIOS had another option for timed startup, so that is what they are using - starts up at a particular time.

counting number of files in folder

While doing server migration, needed a method for counting number of files in folder from the Linux command-line. Found this post, where in the comments this method was listed:
ls -lR | grep ^- | wc -l

capturing DV on old PowerMac G5

Resurrected a retired PowerMac G5 (1.9 Ghz, 3 GB RAM) for playback of Planetarium video. It was not fast enough to play full HD movies at 30 fps - coming to only 15 fps for a 12 Mbps HD video. So, relegated it to DV capturing duties for archive work - Hi8 to DV using a Sony TRV330E Digital8 camera.

iMovie for MacOS X 10.4 Tiger - iMovie HD (obtained with a torrent) seemed to be the last working version - iMovie 8 did not run on this machine.

iMovie was capturing the video as .dv files, and the files were being hidden inside the project "file" which was actually a "Package" - so in Finder you right-click and "Show Package Contents" to see the Media folder inside. To convert .dv to .mov without re-encoding, used MPEG Streamclip.

Edit: Finally this solution was abandoned - the .mov produced in this manner still needed rendering on FCP at the Studio. Then tried capturing in an old version of FCP, that worked without rendering.

stream not working with jplayer

After doing some tests for changing the default player on our website to jplayer, AfriStream stopped working with the jplayer based player, all over the site! But only from my work computer. So, guessed it was some cache problem. Cleared the cache, and it started playing again.

changing the default player on radiosai.org

On the home page of radiosai.org the links to the streams use a flash player from WaveStreaming.com. Trying to substitute it seamlessly using jplayer, found that modifying the size of the player skin requires more CSS skills than I have at the moment. Will think of something later.

Edit: Most problems have a brute-force solution! Resized the images by a factor of half, halved all the pixel values in the css, et voila - a half-sized player! That was too small, so multiplied again by 1.4 to get the size we want.

setting up live video streaming - resource usage

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.

When the Wowza streaming server was serving at around 600 Mbps, that is around a 1000 clients, top reported the following usage of memory and cpu:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           

18825 root    20   0 2009m 1.3g 7892 S  113 17.1   1064:58 java  

Since this is a quad-core machine, probably top reports 100% as maxing out a single core. So, we have plenty of cpu to spare, and could easily max-out the 1 Gbps port. (And that's what we did .... running video at 800 kbps per instance.)

disk full on saiwaves

While uploading files to dl.radiosai.org saiwaves reported disk full - unable to write to disk etc. As given at this post, removed some 3 GB from Trash by doing

rm -Rf ~/.local/share/Trash/*

Later, P removed around 70 GB (!) of old database backups.

mysql database error

Our schedule page and audio search page stopped working - sometimes the pages would respond with "Could not connect to database" and sometimes they would show 0 records returned. Trying to connect with the phpMyAdmin using Plesk gave the error message

MySQL said: Documentation
#1 - Can\'t create/write to file \'#sql_abc_0.MYD\' (Errcode: 17)

Reported it to nettigritty support, they restarted the service, everything became fine.

setting up live video streaming - aspect ratio and timezone

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.

 P wanted the timezone on the server to be set to IST, for ease of recognizing recorded files etc. Did it with dpkg-reconfigure tzdata
which is the Ubuntu way of doing it at present.

Then: the flv video which we saved on the server seemed to have an NTSC flag set somewhere. Probably originating from the Adobe FMLE (Flash Media Live Encoder). So, the video was being seen with a skewed aspect ratio even though it was square pixels 400x224 25 fps. Finally solved it at the client side using stretching:"exactfit" as a parameter for the jwplayer.

setting up live video streaming - recording on the server

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.


There are different ways of recording.
1. using live-record as stream type in Wowza. This does not give us control over when recording starts - recording starts as soon as the source gets connected.
2. using the LiveStreamRecord addon module, we can choose when to start recording and when to stop.

So, we used the latter, set up a password protected page on our website for the record enable form which is included with the module. We set it up not to overwrite (so it would create a new file like myStream.sdp_2.flv etc increasing the number to _3, _4 etc.) Also set it to start on keyframe and record data. (Metadata, I suppose.)

First we tried setting up http pseudo-streaming with mp4 on Apache. After setting up the module for streaming video the seeking was not working properly for parts of the video which had not yet been downloaded - jwplayer was playing the beginning of the video after the seek.

Then the audio and video were coming out of sync on the recorded mp4 file. It looks like many people have faced this issue.

So we tried recording as flv files instead. With that, the audio-video sync problems were solved. But since the module for flv seek over http uses Lighttpd - setting it up was a bit of a pain - we fell back to providing the video over rtmp using Wowza's vod application. That, of course, worked perfectly.

Argument list too long

Doing ls or mv invariably leads to Argument list too long errors. A nice discussion about ways to get around this:
http://www.linuxjournal.com/article/6060

setting up live video streaming - download links

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.

Setting up a download link - I tried to do this by adding the .htaccess file like in dl.radiosai.org with the lines


  ForceType application/octet-stream
  Header set Content-Disposition attachment

But this did not work, even when I modified the config file again, as in
https://help.ubuntu.com/community/EnablingUseOfApacheHtaccessFiles
adding the line

AllowOverride All

So, did it by editing the config file:
http://www.jtricks.com/howto/content_disposition/apache2.html

       

                Options FollowSymLinks
                AllowOverride All
               
                    ForceType application/octet-stream
                    Header set Content-Disposition attachment
               
       

 

Then, files can be either copied or linked into the download directory, and accessing them automatically gets the "Save as" dialog box for the user.

setting up live video streaming - iPad and iPhone

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.

After setting up Wowza server as per my earlier post, tried to get the live stream to work on iOS devices - iPad, iPhone, iPod touch etc.

To make the stream work with iphone, the encoder settings have to be tweaked. Format has to be H.264 and AAC audio. Otherwise the device gives the message, "This video cannot be played."

AAC settings should be low complexity - I found 22050 Hz, mono, 32 Kbps to work OK.
H.264 settings have to be Baseline 3.0
(set by clicking on the spanner icon next to Format: H.264 in Adobe Flash Media Live Encoder)

When this is done, the following url works for iOS devices, when entered into Safari on iPhone, for eg.:
http://server1.radiosai.org:1935/live/myStream.sdp/playlist.m3u8
(entering this URL on a normal computer does not work).

In order to make this work with the jwplayer, which we can embed in a page which will work for desktop browsers as well as iOS devices, the player had to be updated to the latest version - 5.8. Then the sample code for embedding worked fine.

local recording along with live streaming

S was looking for solutions for local recording along with live streaming, for upload to vimeo later. He wanted to explore Quicktime broadcaster for this.

There are several possible solutions, which we can try out.

1. Use a DV hub, record DV on another computer, encode offline after the event, then upload. This will result in best quality.

2. Record on DV tape on the DV VCR which you are using as source, capture and encode after the event. The tape has to start before the stream starts, and the tape has to last for the entire event, because stopping the tape causes the stream to restart.

3. Recording on the same machine which is doing the broadcasting - seems to require some funky workaround like
http://forums.adobe.com/thread/657963
but may be possible. To open multiple instances of FMLE you have to use this technique:
http://osxdaily.com/2011/05/11/multiple-instances-application-mac/

4. Using Quicktime broadcaster instead of FMLE is also probably possible, wowza server supports QT broadcaster also as a source. But broadcaster also seems to have the same limitations - setting up the entire chain for it separately seems to be a waste of effort.


(Finally the solution chosen was 1.)

setting up live video streaming - C++ RTMP server

This will be a series of posts about setting up video streaming on our new dedicated server from Voxel.

First attempt was to replicate my earlier test, and set up the C++ RTMP server. That involved re-compiling from source, since the binary did not seem to run on the Ubuntu machine.

1. Checking cpu type:
cat /proc/cpuinfo shows it is
Intel(R) Xeon(R) CPU X3330 @ 2.66GHz
2. so trying crtmpserver-690-Ubuntu-10.10-i386.tar.gz
3. Procedure as in my earlier test
4. Gave error "no such file or directory" when tried to run the executable.
ia32-libs recommeded by this launchpad post.
5. apt-get install ia32-libs and then tried. Then the executable runs, but seems to have some error, since the console doesn't come up. Tried with the 64 bit binary also, still no go. So, tried the sources from http://www.rtmpd.com/downloads/

6. For that, while following the instructions to install at the rtmpd site.... had to install lots of stuff.

set CMAKE_CXX_COMPILER to a valid compiler path or name -> apt-get install gcc

and

apt-get install g++

OPENSSL_INCLUDE_PATH-NOTFOUND -> as per this forum post,
apt-cache search libssl
grep SSL

Then I installed apt-get install libssl-dev

Finally worked. Binary is in /root/crtmpserver-690/builders/cmake/crtmpserver folder.

But....
C++ RTMP server does not have the nifty features of Wowza, does not automatically make the http  pseudo-streaming chunks for streaming to iPad etc. So, decided to go for Wowza server after all.

google analytics code

Upgrading to the new template on blogger had removed the GA code. Added it back today, going to Edit HTML in the template editing section, and adding the code just above the /body end tag.

Actually a related topic, checking if the analytics code is working:
http://www.rachaelarnold.com/dev/archive/how-to-check-if-google-analytics-click-tracking-is-triggered
this method uses  Firebug, with the Net panel enabled. We should be able to see requests starting with _utm.gif? - and the params tab gives the info being sent to Google.

streaming video and quality

Just a small study I did about streaming video and quality.

Ted talk quality, for eg.
www.ted.com/talks/ken_robinson_says_schools_kill_creativity.html

- wireshark clocked it at 0.415 Mbps - let's say 400 kbps.

(Ted uses variable bit rate streaming - bitrate changes with scene as well as with bandwidth availability)

Youtube video at 360p -
http://www.youtube.com/watch?v=I1Tbn1GtIGE&feature=grec_index

21.6 MB for 8 minutes 45 sec
21.6 MB/525 seconds
= 0.0411 MB/s
= 0.329 Mbps, close to 300 kbps.

But these are all offline encoded videos. For live encoder, quality will be somewhat lower for the same bitrates.