free space on colinux and hard disk performance

Over the last two days, there were several interruptions on our streams. Problem seems to be excessive Hard Disk usage, since copying files was taking a very long time. But at a time when the system was behaving normally, trying out the counters
1. PhysicalDisk: Avg. Read Queue Length Should be less than 2
2. PhysicalDisk: Avg. Write Queue Length Should be less than 2
3. PhysicalDisk: % Disk Time more than 50% indicates a bottleneck
as given here indicates that the disk is fine. That page says

to ensure that it was not a processor or memory bottleneck, I also recorded % processor time and available bytes. As you can see from Diagram 1, the processor's average was below 30%. If the processor were the bottleneck the trace would be over 80%. On the other hand, if there was a memory shortage, available bytes should drop below 10MB.

And our processor usage was < 20%, with > 200 MB of memory also available.

Maybe I should try the same checks when the system is crawling. If there is some indication of what application is causing the hard disk usage in some log somewhere, it would have been useful. I suspect IIS, since SSSBPT and the audio downloads are also on this machine.

Checking the colinux virtual machine, found only 100 MB free, 10%. Cleaned up to make it around 40%, 400 MB or so, by the following.

apt-get clean cleaning out /var/cache/apt

As given here, postsuper -d ALL
for /var/spool/postfix

from /home/sgh/playlist,
ls ????09* to find the 2009 playlists, removed them and older ones...

Some interesting info:
2007 Feb asia/africa/america started
2004 Oct started web stream

for /home/sgh/log

mv 0*.check ../audio/log_backup 2> /dev/null
mv 1*.check ../audio/log_backup 2> /dev/null
mv 2*.check ../audio/log_backup 2> /dev/null
etc - the errors to dev null, because mv complains that it cannot keep the same user ownership and permissions.

tail -5 logfile > logfile did not work, so had to do
tail ices.log > icesnew.log
mv icesnew.log ices.log
after taking backup.

But tho playout continues, log file is not appended :(
only after playout restarts next day? Anyway, had to restart all playouts by restarting colinux service since the streams started skipping, so the logs have restarted now.

midi works tho' audio is dead

The Yamaha AW4416 came back some weeks back after a stint at Chennai, with no improvement. Apparently it can't be fixed without a high cost. The audio output is dead, and I believe the inputs are dead too. Some of the switches are also not working. Tested the midi connections today by connecting to laptop like in my previous post through a Tascam US-144. The Midi section seems to be fine. They seem to be getting a firewire mixer at the Studio. Then maybe I will get the Tascam US-1641 once again.

hardware debugging - cordless door-bell

A cordless door-bell which K got for me had a funny problem. When I affix the remote switch on our door, it would not work. Remove the switch from the door, and it works, though it is at the same distance from the bell. When KR fixed it to the wall instead, it worked. So, in retrospect, it was probably some sort of capacitive effect from the aluminium door frame acting through the plastic case which was changing the transmission frequency or something.

asp error '8004020f'

BT subscription feedback form returned this error. The many-headed say this means that the e-mail was rejected by the server for some reason. Checked the asp code, it had
Fields("http://schemas.microsoft.com/cdo/configuration/smtpserver") = "localhost"
which listed the wrong smtp server. Corrected it and it worked fine. A wrong email address, as mentioned here, leads to asp error '8004020e'.

shopping cart code and digital downloads

A google ad on this very blog pointed to www.fatfreecart.com which looks pretty useful. Free shopping cart code for google checkout and paypal. Of course, many hosting companies offer free shopping cart code with their hosting too... And DigiVendor seems to be a useful MySql-Php secure downloads payment option. The Google Checkout API docs detail the process, and provides sample code, too. Similar stuff exists for paypal.

ebook reader for Nokia 6303

Struggled quite a bit for helping V find an ebook reader for his Nokia 6303 classic. C mentioned that it was an S40 phone and not an S60 phone. That helped the google searches. Mobipocket's java version had some signing bug which was not allowing it to work properly. I sent him this list,

ReadManiac

Libris is not free, $10

Wattpad reader
free, but supposed to read books from their website, I don't know if you can read your own files.

Wikitome reader seems to be for Series 40, so may work on 6303

Again, getting books in that format?

And he replied with his solution, MTextReader, which seems much better, and free too.

disabling google buzz in a corporate or academic network

R wanted some way to block google buzz at work. According to him, too many people were spending too much time on Buzz, with a spike in network bandwidth. And following friends of friends and other such undesirable activities. This was my reply.

This question has been asked in gmail support forums also, but since
buzz is integrated into gmail, it seems difficult to block it using
traditional methods. Here is one way I can think of:

Make the user log in to gmail using an older browser, or a browser not
supported in the latest gmail version. For example, I am typing this
in Opera 8.53 - available from oldversion.com - then, buzz is not even
seen. It is not using html view, either - it is using the older
version of gmail, which you can see from the url
https://mail.google.com/mail/?shva=1&ui=1&ov=0
This version has most of the useful things like address completion and so on.

(If you can redirect users to this url when they go to gmail also, it
might do the trick.)

One way of forcing users to use an old browser would be to allow gmail
only on terminal sessions running on a terminal server or remote X
sessions running on a linux server or remote desktop sessions. Since
the user is just logging on to the server, permissions can be set so
that the user cannot install any sw. Possibly such permissions can be
set on the desktops also.

Edit: R says it's not practical for him to change user settings. He will explore a url rewrite using squid's php rewrite or something like that, to redirect any requests to mail.google.com/mail to mail.google.com/mail/?ui=1

malware and more

Once again, malware hit some of our websites. Had to do stuff from this earlier post. These were coming in via ftp from remote hosts. So, ftp logs showing logins from Poland, Bulgaria, etc were red flags. All sorts of stuff, like iframes and links to various sites which were themselves hacked, and so on. Some examples of malicious links, do not visit these!!!
videointerviewtips.com/pdf/free-resources-old.php

and in iframes

jL.chura.pl/rc/

Luckily Google and stopbadware.org have filters in place, warning us. But some sort of pro-active thing should be done, instead of reactive, like this time.

pdf to text

Foxit reader has "Save as text", but only for the pro version. Doubtless there are easier ways to do this, but for the quick and dirty fix without having to install any software, pdftextonline.com does the job. Of course, only those pdfs which have text in them (and not just images) will be useful with this text extraction.

testing Tata Indicom photon plus usb wireless connection

First I tried the McAfee speed test like in my previous Airtel test and it showed 200 kbps.



But then I tried connecting to our studio ftp server, and it started serving at 1500 kbps! This sustained for quite some time, not some caching effect. So, their international ports are probably slow. Again, their speeds seem to fluctuate. Tata Indicom's own test showed Your current download speed is: 695.50kbps

It was able to connect to our colinux machine, so no problems with ssh or rdp being blocked or anything like that. Latency-wise, it seemed even better than our leased line Tata internet port! 186 ms on pinging google.com, versus 256 ms with the leased line. Similar results to tracert as given here.

My tests were on Windows XP, so the built-in drivers could be used. The USB device has the drivers on in-built memory which is recognised as a CD drive. Autorun.exe and all that. For Linux, these people seem to have it worked out. Basically using wvdial, copy into /etc/wvdial.conf

[Dialer Defaults]
Modem = /dev/ttyUSB0
Init1 = ATZ
Phone = #777
Username = internet
Password = internet
New PPPD = yes
Stupid Mode = 1

then sudo wvdial

Edit: Some more info. The device gets a bit hot like a mobile phone during talk-time! Bittorrent speed tested with an Ubuntu CD download, was around 1.5 Mbps again. So, that seems to be the fastest it can deliver. Upload speeds were slightly slower - 800 kbps for the local ftp server. This product is a USB-based one, Tata also has a router configuration. Tariff is somewhat higher than BSNL's wireline ones.

mixer glitches

The Tascam TM-D4000 I'm currently using after the Yamaha 4416 went back to Chennai for repairs, showed its temperamental nature a couple of days back on Tuesday, when it "hung" with blank display and saturated output - all inputs set to +10 dB or something like that. This happened 10 minutes after switching it on around 4 pm. So, the reason S used to report this in the Studio is probably not water condensation. Turning it off and on does not work, have to wait an hour or so. After an hour, turning it on - worked, again hanging after 7 minutes. Turned it off till Wednesday, then it worked fine. No problems yesterday and this morning, either. But must be ready with a backup plan, since this sort of failure is a possibility.

trying out Java streaming radio apps

Tried out various Java apps listed on the radiosai website on my Windows Mobile 5 phone which has a Java environment also, but all of them failed to play. At least VirtualRadio gave an error message:
creating player:
javax.microedition.media.MediaException
NULL
I suppose that is because the Java environment does not have access to an mp3 decoder.

gprs counter for mobile - WM5

Tried out the built-in gprs data counter in HomeScreen PlusPlus, the settings for which which can be accessed via Start -> Settings -> Personal -> Today -> Items -> HomeScreen PlusPlus and clicking the Options button. Or by pressing and holding near the cpu icon.



But for some reason it did not work on my phone. Maybe I should have tried rebooting the phone after setting it or something. But anyway, it showed 0 incoming 0 outgoing even after using nearly an MB - Rs 5.40 worth - by just seeing half a page of slashdot in skyfire!

some fixes for windows media encoding

Some of the VOB files to be encoded showed some audio delay when processed through VirtualdubMod even though they played perfectly well with KMPlayer. Then found this page which explains how to do it with Virtualdub, and this one for VirtualdubMod. Basically,
Streams -> Stream List -> Right-click the audio -> Interleaving -> Delay audio track by xxx ms.
500 ms worked well for me. Virtualdub has the same thing under
Audio -> Interleaving

Then trying to do WME batch encoding as given in this post, finally ended up making a batch file with
cscript.exe wmcmd.vbs -wme "E:\wmv\2.wme"
cscript.exe wmcmd.vbs -wme "E:\wmv\3.wme"
cscript.exe wmcmd.vbs -wme "E:\wmv\4.wme"
cscript.exe wmcmd.vbs -wme "E:\wmv\5.wme"
and so on, with each wme file having a separate input and output file specified. The input file is in two places in the wme file, for audio and video. The wme files are xml files, so a text editor can be used to generate them by modifying older wme files.

But in typical Microsoft fashion, the resultant output files are not exactly the same as the files generated using the WME gui. Because, WMVAppendGui cannot join these files with files produced earlier with the WME gui! Have to re-create those files also with the wmcmd.vbs commandline technique.

BleachBit

From this slashdot story about tracking, went on to EFF's Panopticlick and bleachbit. Bleachbit - removes cache/temp/cookies etc from a variety of programs and plugins including Flash cookies and Java cache - seems quite impressive, a portable version too, along with Windows and Linux versions. Saves space on C drive for the space-constrained, too.

delay in streams and jitter

Today, I sat down to quantify the amount of delay in our stream which is being aired now on AsiaStar also. I noticed that most programs on AsiaStar were starting a few minutes after the hour - like 1.05 instead of 1.00 etc. Checked if the time on the server was correct, yes, it is set to automatically keep the correct time using updates from time.windows.com

Then checked the log to see when files are being played:
tail /home/sgh/ices/asiastream/ices.log
This does not have any timestamps. But
ls -l /home/sgh/ices/asiastream/ices.log
gives last write timestamp, saw this was 13:04

Checked the playlist on xmms, according to the xmms duration listing, it should have been 13:03. So, this adds some jitter.

Next, there is the buffering time in the ShoutCast server, and also buffering in the client. This probably adds a few more seconds. Checked the time gap from the time on the log and the time I heard it on AsiaStar - 13:20 on log for next file, I heard it at 13:23. This is probably the buffering on the server + client.

Client buffering probably changes with time. For example: I played AsiaStream on my machine, it was approx 10 seconds later than AsiaStar which is AsiaStream coming after going up and down from the satellite. The satellite round-trip takes approx 2 seconds, so my machine was 12 sec behind the Melbourne machine.

permission error

My colleague got this error from the webserver:
Microsoft OLE DB Provider for ODBC Drivers error '80004005'
[Microsoft][ODBC Excel Driver] Operation must use an updateable query.

Googling found these possible causes, and changing permissions for the folder seems to have solved the issue. Allowed the user called IUSR_WEB_SERVER_NAME (the web server) to write in that folder.

Windows media encoding and virtualdubmod batch jobs

Investigating batch converstion with Windows media encoder, found this resource which includes info on the command-line encoder. This page has the info in a more readable form - just
cscript.exe c:\path\to\wmcmd.vbs -loadprofile encoding-profile.prx -input "F:\input-folder" -output "F:\output-folder"
This page explains how to get the prx files from the WMEncoder, Properties -> Compression -> Edit -> Export button. But of course, it did not work for me,
.Error occurred in transcoding: Error Code = 0xC00D002F
since my WME does not work with Xvid encoded files, needs a nominal crop of a couple of pixels before it works. Probably then it uses a different input codec or whatever.

But checking out cscript.exe wmcmd.vbs /? found that it supports WMEncode wme session files too! So next time I can try that.

Virtualdubmod also has a batch encoding option with input and output folders, as given at this page and I used it this time: File -> Job Control, and Edit -> Process Directory from the Job control window.

machine translation of Indian languages

This IIT-built automatic translation engine is interesting, pointed out to me by BK. If you don't prefer the Hindi keyboard, you can use other tools like gmail's or google's or others like writeka, then copy-paste into the duur sampark page. The attached sentence took nearly a minute to translate, and it seems to be highly Hyderabadi Telugu! (Click on the picture below for larger image).

voddler

Voddler seemed a good service, for watching ad-supported movies. But it really needed a 2.5 Mbps connection, so not suitable for us - too much time spent buffering. The interface is a bit quirky too, since they have made it "remote-friendly" for people running media center PCs or whatever, but PC unfriendly, not allowing mouse-clicks! And officially only open for people in Sweden, of course.