compiling vlc (or vlc-warp) for Windows

Could not find any relevant solution to the cross-compiling issue - the dlls etc have not been made properly, so even mounting the Linux partition on Windows and trying to run vlc fails. Edit - the cross compiling post has now been updated with the solution - installing wine-dev package.

Tried the Msys method of compiling VLC on Windows. That fails when trying to build automake etc, saying
configure: error: cannot run C compiled programs

Next was to try the "advanced instructions" at  https://wiki.videolan.org/Win32CompileMSYSNew/
That has a note halfway down, saying

Note: I found that compiling 2.1.0 requires xgettext, msgmerge, msgfmt and so on. These in turn require libiconv. I'm not sure if that suffices as at this point I gave up. 

So I also gave up, for now.

cross compiling VLC Warp for windows

VLC wiki recommends cross-compiling from Linux as the preferred route to compile for Windows. But trying to compile the vlc-warp cloned from github, ran into lots of dependency problems. My Kubuntu 12.04 was borked by installing .deb which depended on newer libc versions, so gave it up and installed Ubuntu 13.10. 13.10 initially had older libraries like libc and so on, but after updating, the libraries required were updated to newer ones.

Then, followed the Win32 cross compile guide on the VLC wiki, and the build went through. I followed the "Preferred method" of MingW on Linux. First installed the build tools from the Unix compile wiki,

sudo apt-get install git libtool build-essential pkg-config autoconf

and then did apt-get install for all the packages mentioned on the cross compile guide. I used a tar.gz with a backup of the git clone of the vlc-warp directory which I had got for the Linux compile, and the backup of the vlc-contrib-i686-w64-mingw32-latest.tar.bz2 pre-built contribs which were got earlier while doing make prebuilt for the contribs. The make itself took only around 20 minutes, just like for the Linux make.

Unfortunately, after doing the make,
make package-win-common
failed.

make[3]: *** No rule to make target `/usr/include/wine/windows/stdole2.idl', needed by `stdole2.tlb'.  Stop.
make[3]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc/activex'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc'
make: *** [stamp-npapi-build] Error 2

I guess a solution to this would have to be a separate post.

Edit on 6 Jan 2014: The comments below helped solve this issue. Installing wine-dev package solved it.

clearing up disk space and increasing size of loop drive

My Kubuntu install, upgraded to 12.04 from 10.04, was running out of disk space on root. It was installed on a root.disk file on Windows NTFS partition. First tried googling and found this post, so did

sudo apt-get clean
sudo apt-get autoclean

sudo apt-get autoremove  

sudo apt-get install localepurge

Also tried listing packages with

dpkg-query -W --showformat='${Installed-Size} ${Package}\n' | sort -nr | less

and removing those which I did not need. But still, free space on disk was only less than a GB, compiling was eating up space too. So, tried googling increase size of Linux loop drive, and found this forum thread. That had a lot of "dangerous" warnings. Lost track of it since it was on the Mac, saw the Mac Safari history as given at this post - the History.plist file was a binary file, opened using PsPad's Hex editor.... Then found this solution by googling increase size of /dev/loop0

Booted into Ubuntu 13.10 Live DVD, navigated to the root.disk file.

sudo dd if=/dev/zero bs=1MiB of=root.disk conv=notrunc oflag=append count=1000
(where count is the number of MiB to add)

Then mounted it with
sudo losetup /dev/loop1 root.disk
and resized with
sudo losetup -c /dev/loop1
sudo resize2fs  /dev/loop1

At this point, the system alerts me to
Please run e2fsck -f  /dev/loop1
So I do a
sudo  e2fsck -f  /dev/loop1
and then repeat the
sudo resize2fs  /dev/loop1

Et voila! Done.


Only now did I see the official Wubi guide, which gives a solution to this very problem, in a "safe" way.

screenshots on Linux

Found this page with a review of lots of methods to take screenshots,
http://tips.webdesign10.com/how-to-take-a-screenshot-with-ubuntu-linux

 sleep 10; import -window root MyScreenshot2.png

seems to be the least resource hungry method, since it needs only ImageMagick and not kde etc.

Taking screenshots with vlc, the vlc screen seems to come on top of other windows like options window etc. Or the window ordering for screenshots is different or something like that. Need to arrange open windows in non-overlapping ways to avoid problems.

compiling vlc-warp from git

Paul Bourke has put his vlc-warp on github, tried compiling it for Linux. Before vlc-warp, wanted to check if vlc 2.1 would compile. Downloaded from videolan.org and started following the instructions. (Initially, I tried without reading the instructions, quickly ran into dependency issues like the one discussed here.)

Tried it first on my dual boot Kubuntu 10.04. But trying to get the dependencies using
sudo apt-get build-dep vlc
did not go through - the package versions of the dependencies in 10.04 were too old. So, decided to try it using a Live DVD, Ubuntu Studio 13.10 was available with a colleague with his OSFY magazine subscription.

Initially, tried with the vlc source downloaded onto a Windows FAT32 drive. There, ./configure was not executable - apparently, the drive was mounted noexec or something like that. So, copied the source to the home folder, and started again.

Next error was, "Unable to find a source package for vlc" when trying the
sudo apt-get build-dep vlc
For this, selected both the "source files" and "universe" repositories in the package manager, did
apt-get update, then tried
sudo apt-get build-dep vlc
again, worked now.

./configure took a couple of minutes to run, and make took around 20 minutes on my machine, with an AMD Athlon dual-core 1.9 GHz processor. At the very end, make gave a couple of fatal errors, saying make-alias did not work. Most probably this was due to copying over the source from the Windows drive, which made all the files including make-alias and configure lose their executable permissions. But the only issue was that there was no vlc binary in the root folder, I had to go to the bin folder and choose the vlc-static binary. That worked. And I could choose the OpenGL video output - that is critical for vlc-warp to work.

Then tried with the actual vlc-warp. First tried downloading the source as a zip file using the link given on github. That did not work - make gave errors saying no git repository found or something like that. Then did it properly using
git clone https://github.com/pdbourke/vlc-warp-2.1.git
as given in the VLC build instructions. Then,
./bootstrap
./configure
make
worked fine, twenty minutes later, I had the binary. Tried it out with a standard mesh file from Paul, performance was similar to the Windows version.

In summary: first check with VLC 2.1 installed on your machine if OpenGL video output is available, and if it works. Only if that works will vlc-warp 2.1 work. If your machine does not have a supported graphics card, the OpenGL option either won't be available, or using that option won't get you video using vlc. In that case, vlc-warp won't work either.

Edit: More caveats!

phplist unable to recognize bounce if email id has space before it

Looking at the bounce messages on phplist, found a couple of bounces every day, for which phplist could not identify the list subscriber - listed as Unknown. Viewing the bounce, found that the email id had a space in front of it, which was the reason why phplist could not identify it. Unsubscribed manually. (This is phplist version 2.10.10).

previewing video in Reaper, workaround for AAC

My version of Reaper, 3.52, had problems displaying videos using the FFMPEG option. Tried downloading various versions of the ffmpeg dlls and put them in the Program Files\REAPER folder with no results:

Finally deselected the FFMPEG option, choosing to preview video only with DirectShow - that works fine for me now. Options -> Preferences -> Media -> Video/REX/Misc, Disable FFmpeg decoding.

Most mp4 files have AAC audio. Unfortunately, Reaper (at least my version) seems to have issues with AAC, audio stutters and finally reduces to a single repeated hum after a few seconds. Workaround was to extract audio using Avidemux, save it separately using Audio -> Save Audio - and use that audio in Reaper.

file permissions

Just a note about file permissions with SMB on Linux. P had moved some folders to another folder. As a standard user with ssh, I couldn't delete them - had to be root. P then demonstrated that the folder with files in a Samba share was with owner nobody:nobody so that it could be deleted by users accessing the Samba share as guest on Win or Mac.

quiz code correction

A found a bug in the 9th question of a quiz featured as a Sai Inspires link, which had been uploaded to our website in 2009! The answer was not being displayed. I checked for the reason in javascript, did not find any problem. P found the reason - the "answer area" div had been omitted in the html!

network congestion and the nuclear option

Before starting the evening live broadcast today, listening to AsiaStream took a long time to buffer. Checking cyberoam usage logs, found that WAN download was maxing out. Even the firewall rule was not helping, probably because the connections were made before 4.15 pm, and the firewall rule after 4.15 pm would affect only new connections.

Under these conditions, BUTT would not connect easily. Earlier, I would just restart cyberoam (soft restart) and that would kill all concurrent connections, allowing BUTT to connect.

This time, went into Cyberoam -> System -> Diagnostics -> Active Connections list, and killed around 10-15 of the around 400 active connections, which had used max download bandwidth. Most of them were port 443 - https - most probably youtube. One or two were chat ports 5222 etc. Then was able to start streaming without issues.

ftp speed bump - restart solved

Our internal ftp server which also hosts our mailing lists suddenly slowed down: transfers which went at 100 Mbps over the LAN started going at 3-4 Mbps. A restart of the machine solved the issue. But during the restart, P reports, the system required a manual execution of fsck. Once that was done, system booted. It was our other internal server which needed the manual choice of boot device.

email for a new domain - options

Copy-pasting an email I sent about adding email for a new domain:

I see that the mail exchanger (MX) records are not set for this domain. So, currently, no emails for this domain are configured.

There are many options for email.

1. use the built-in free nettigritty email, or do a mail forwarding to some other account. This may be easiest to configure, but I have not tested nettigritty's mail servers reliability.

2. Use paid mail hosting, like google apps for business which starts from Rs. 150 per user per month. There are, of course, many other options here.

3. Unfortunately google's free plans no longer allow new sign-ups, so the option we chose for our earlier domains is not available. But Windows Live for domains is available for free. We could use that, and if necessary, forward to a gmail / sssbpt.org id also.

4. There is probably one more option - domain alias. Google apps allows you to have aliases, so that email to webmaster@ournewdomain.com can be directed to webmaster@ourolddomain.org etc. 

youtube download failure and solution

Yesterday, for the first time, the youtube mp4 file I downloaded using Flashgot Firefox add-on did not work. Then, another file I downloaded also appeared to be "corrupted". Googling got me this page which indicated that the problem was that youtube was using MPEG-DASH streaming, and so the files were not downloaded correctly.

Tried using Youtube Center as indicated in this Dave's tip, but even with the Youtube Center firefox add-on, when downloaded with flashgot, the downloaded file still had issues.

Then tried the old faithful - the KeepVid.com bookmarklet. Finally, video downloaded with that worked fine. Of course, a caution for keepvid.com users, you have to click on the download link on the right of the text box and not on the bigger "Free Download" link below, which is actually an ad for another downloader product. And keepvid needs to run a java applet.

What could I do with the "corrupted" downloaded videos? This post indicated that GPAC might be able to help. (BTW, Nirsoft's freeware tools are really nice!)

And indeed, the mp4box utility contained in the GPAC download did the trick - made the video readable by AVIDemux using

MP4Box -add file.mp4 out_file.mp4

Even though the documentation said that the -isma switch would make the video compatible with most players,  doing
MP4Box -isma file.mp4

had a similar result as the -add above, with the file being readable by VLC and AVIDemux, but not directshow. Used Avidemux to re-encode the file as HuffYUV, then directshow could read it. Using Avidemux in copy mode was not enough. And after all this, there was no audio stream. Apparently, the downloaded file contained only the video stream and not the audio.

So, the best option would be to re-download using Keepvid or something like that, unless you need only the images and not the sound, like in my case.

Edit: Just noting that mkv files produced in "Copy" mode using Avidemux could be played by TheKMplayer, VLC etc, but not directshow/Windows Media Player. avi files produced in the same way could not be played by TheKMPlayer either, though VLC could.

side-chaining - ducking in Reaper

Following the method given at AskAudio:

1. Drag and drop from the io button of the track which is going to be the controlling track, to the track which is the one which will be affected by the compressor.
2. Choose drop-down and change the destination of the receive to 3-4 instead of the default 1-2.
   
   
   
   
   
3. Choose the Detector Input on the compressor to be Auxiliary Input L+R and other settings as per your choice.
   
   
   

shortcut to creating multiple videos on dome

I wanted to have a number of videos tiled on the dome, maybe six or nine videos in the front upper surface of our tilted dome. The classical method would have been to use blender and a video texture. But that is very time consuming, and my blender skills are extremely rusty. Looking around for shortcuts, the General quadrilateral transform in Virtualdub was the first thing I thought of. But that filter does not support overlays - background remains a single colour. The Reform and Reformer plugins for avisynth did support overlay, but..... Reformer turned out to be buggy, the background video kept moving forward and back and turning green etc.... and Reform's output was too jaggy and aliased.

So, used the General Quadrilateral transform in Vdub, created a mask for each video, and used simple overlay calls in an avisynth script to overlay all the videos.

lrmask=Imagesource("lower_right_white0.jpg",end=450)
lr=avisource("lower_right.avi")

and so on, where lower_right_white0.jpg looked like 

bg=overlay(bg,ul,mask=ulmask)
bg=overlay(bg,ll,mask=llmask)
bg=overlay(bg,um,mask=ummask)
bg=overlay(bg,ur,mask=urmask)
overlay(bg,lr,mask=lrmask)

For getting the shapes right, first I did a trial and error using the warped mesh as a template. But that was not getting me the shapes so easily - the warped mesh has latitudinal and longitudinal lines, not rectangular lines. Not so suitable for placing rectangular videos. Finally, thought of directly painting with gimp on the dome - put the mesh as a background layer for aligning the frame to the dome approximately, then painted the desired rectangles on a new layer. Using those rough outlines, noted the vertex co-ordinates as below.

lower middle video = 440x248 - 1.774
lbotx= 740
ltopx= 740
lboty=1079 #default
rboty=1079 #default
rtopx=1180
rbotx=1180
rtopy= 832
ltopy= 832

upper middle video = 340x220 - 1.54
lbotx= 800
ltopx= 800
lboty= 740  
rboty= 740  
rtopx=1140
rbotx=1140
rtopy= 520
ltopy= 520

lower right video 
lbotx=1264
ltopx=1264
lboty=1079 #default  
rboty=1079 #default  
rtopx=1560 #changed from 1600
rbotx=1600
rtopy= 780
ltopy= 832

calculated lower left
rbotx= 656
rtopx= 656
rboty=1079 #default  
lboty=1079 #default  
ltopx= 360 #changed from 320
lbotx= 320
ltopy= 780
rtopy= 832

upper right video
lbotx=1260
ltopx=1220
lboty= 770  
rboty= 730  
rtopx=1480
rbotx=1550 #changed from 1590
rtopy= 530
ltopy= 600

calculated upper left
rbotx= 660
rtopx= 700
rboty= 770  
lboty= 730  
ltopx= 440
lbotx= 370 #changed from 330
ltopy= 530
rtopy= 600

updating remote database with records in one field

We had to update the 'firstplayed' field of the schedule page database on our website. Our local database had the info. The way to do this seemed to be:

• Export the filename/firstplayed fields from remote db as csv, 
• import into local db as a table, create a view (run a join query) to get the desired firstplayed field for each filename, 
• export as csv, 
• import into remote db, 
• run an update query there. 

But Murphy's law was evident at every step. My database guru roommate Nz helped out to navigate the Murphy minefield. Remote db was MySql, with phpMyAdmin front-end. Local db was Postgres, with phpPgAdmin front end. The import button comes up in phpPgAdmin while viewing a particular table.

• phpMyAdmin's default csv output had ; (semicolon) as default separator, phpPgAdmin's import needed , (comma) as separator. 
• The remote db often took a long time to respond since it is on shared hosting. 131 seconds to update 2000 records, for example.
• phpPgAdmin demanded the field names as the first row of the csv
• phpMyAdmin ignored the field names as the first row, took that row as data to be entered, and mixed up the fields - firstplayed was entered as filename and vice versa!
• Since there were many streams, the firstplayed from the local db had multiple matches, quite a few with NULL. Just ignored them, and used a NOT NULL condition. Syntax for postgres - IS NOT NULL - not not equal NULL. :)
• Had to do a left() firstplayed field, since the local db had a full timestamp, while remote db needed only the date. 
• Telugustream has a lot of files yet to be updated in the temp_play_history table. So will have to do the above procedure again.
• Finally, the update query syntax for mysql was non-standard - Access style, as my db guru roommate mentioned - instead of update dt set df=sf from dt,st where d.jf=s.jf, the syntax was
  update `dt`,`st` set `dt`.`df`  = left(`st`.`sf`,10) where `dt`.`df` = `st`.`sf`

terminal credentials in the new internap portal

After Internap took over Voxel, the configuration portal was "upgraded". In the new portal, the ssh terminal credentials are shown when  you click on your.servername.com, scroll down to device access, and click on the show link next to the ssh password. The confusing character in the middle of our password is capital i (which might have been small L or the number one).

Sureyyasoft Shira Player mini review update

In my earlier post about the Shiraplayer, I had mentioned the warp quality issues it carried forward from its stellarium base. Now the developer has added a better warp functionality in the latest release 1.5.1 - a custom warp data file module.

Initially, on using this version, I got blockiness, as in the screenshot below.

The developer suggested I restart the application after choosing the warp file, and try again, this time it worked fine. So, no more warp quality issues.

Unfortunately my PC (Athlon 1.9 GHz laptop) is not fast enough for realtime usage of shiraplayer with warping.

The Shiraplayer process with the new warping mode takes approximately as much cpu as the warping vlc player produced by Paul. But Shiraplayer has two processes running, one with the video and one with the other features. The process with the other features takes up nearly 50% cpu on my system. So, Shiraplayer stutters while playing even those videos which play OK on Paul's realtime player. For eg, even the 1.2K videos which take up 30% cpu on Paul's player, playing back at full 30 fps, are played back only at 15 fps on Shiraplayer.

But probably all these will not be an issue for those with more powerful PCs. Probably a decently specced machine from the last two years should be able to handle it just fine.

video editing with After Effects

Since the blender methods for cross-fades were quite cumbersome, tried out After Effects instead. What I have is AE 4.1. For fades, need to set opacity keyframes for the upper track. For previewing audio, have to scrub the timeline with Ctrl pressed. Audio doesn't play when the time controls are used. Menu items Composition -> Preview -> RAM preview or Audio preview can make the audio play. Will try this, check if this is better than installing my old copy of Adobe Premiere. 

video editing with blender

Following the steps given at this post. Basically, choose Video Editing in the Choose Screen Layout drop-down just to the right of the menus, and change the default Graph editor window into a Properties windows when you want to do the export.

Right-click and drag to move clips. K to cut. Shift Right-click on two clips to select both of them, then choose bottom menu item Add -> Effect Strip... -> Gamma Cross to cross fade. Have to enable AV sync if needed, default is no sync.

And as this tip says, to move a view-port to a different monitor, shift+left click on the upper right corner of the view port to make it a separate window.

To get rid of a view port, as mentioned here, move the cursor near the border with the next area till it becomes a double-sided arrow, right-click and choose join areas.

The scroll bar handles can be used to expand/contract the timeline view in the Video Sequence editor.

Mouse scroll wheel also zooms in and out. Middle mouse button to pan. Full details are at the Blender wiki.

Select a clip and hit k to cut. Right-click and drag on the arrows at either end of clip to trim.

another audio cleanup tool-chain - Reaper for noise reduction

In previous posts, I've mentioned using Cool Edit Pro and Sound Forge for audio cleanup. For audio with a lot of hiss, noise reduction with Cool Edit Pro produces a lot of artifacts. Reaper's ReaFir filter in Subtract mode is useful in such situations. For an old recording taken from a VHS tape, the following steps were used.

NR and gating in Reaper,

Then sweetening the sound using the EQ setting "30-band Punch and Sparkle" in Cool Edit Pro,

and finally RMS normalizing to -10 dB RMS with Sound Forge, choosing regions of roughly equal loudness.

Edit: 13 Feb 2020 - adding a couple of videos to demonstrate choosing regions of equal loudness by visual inspection.

and then normalizing them with keyboard shortcut Ctrl Y to speed up the process.  

taking screenshots on Mac OS X

This wikihow article has seven methods of taking Mac OS X screenshots.

Command-Shift-4  -> screenshot region of interest, saved on desktop

Command-Shift-3  -> grab full screen, saved on desktop

Command-Shift-4-Spacebar  -> grab an open window, by clicking on it, saved on desktop.

Adding Ctrl like Command-Ctrl-Shift-3 etc copies to clipboard instead.

Applications -> Utilities -> Grab application is also available. Some extra tools are also mentioned.

more details of Reaper-BUTT broadcast setup

My post on the setup for live broadcasts using Jack OS X was a bit skimpy on the Reaper setup details. These screenshots below tell the story of the setup on the Reaper side.

I've got sends going to a full-mix recording track as well as the broadcast track, so that I can use them without interruption even when I 'solo' any of the tracks to listen to that mic - the 'solo' happens by muting all other tracks in the mix which goes to the headphone (outputs 1-2). The mix which is recorded goes to outputs 3-4. I also have another Jack setup where output 4 gets a compressed signal from the broadcast channel below - which could be sent to the video team, for example.

On this broadcast track, I have a high pass filter to cut out rumble, knee at 100 Hz, an aggressive compressor, and the insert from Jack. The insert's return is used as input for BUTT using Jack routing.

VLC-based Warp-on-the-fly player from Paul Bourke - mini review

Paul has now released the VLC-based Warp-on-the-fly player. Here was my mini-review for the beta test done on Oct 4.

As Paul mentions, there are some performance issues. Relatively low bitrate / lower resolution files are OK. The sample1.mov file  - 1280x1280, 6.4 Mbps with avc1 codec - plays smoothly. A 2K file I encoded with Xvid codec, 9.8 Mbps, plays at something like 15 fps, with more frames being dropped where the scenes are more complex. Seems to be CPU limited in my case, so people with beefier CPUs should be OK. Mine is an Athlon 64 X2 dual core processor at 1.9 GHz, with Nvidia Geforce 9100 M Graphics.

CPU usage -

1280x1280, 6.4 Mbps with avc1 codec with warp - 55% with beginning peak of 75%

1280x1280, 6.4 Mbps with avc1 codec without warp - 40%

2048 x 2048 Xvid codec, 9.8 Mbps with warp - hits 99, 100%, hence frames dropped to 15 fps or so.

2048 x 2048 Xvid codec, 9.8 Mbps without warp - 65%, with peaks of 75% for complex scenes.

Another issue for me is that the snapshot feature, or the convert feature, does not capture the warped file, but instead captures the input frame. Perhaps there is some setting which I should do in order to capture / export the warped frames from this player? If I can get the capture / export working, then the performance issues will be moot......

Sureyyasoft Shira Player mini review

Tried out the Shira Player from Sureyyasoft - a player modified from a Stellarium base. At first it crashed, the developer sent me a shader file to put in the data folder, after which it works. Interestingly, it does on the fly warping of fisheye movies using the spheric mirror distortion built into Stellarium. But as mentioned in my previous post on Stellarium, the output image quality when warped is not so great, especially when I use image capture and compare with the other warping tools.

Image obtained from the avisynth plugin, using Flipvertical():

Image obtained from Shiraplayer:

Additionally, there seems to be some sort of up-down flipping involved: Flipvertical() was used in the avisynth script to get a similar image.

Update: Newer version of Shiraplayer with these issues solved, discussed here.

infection on Studio network

Our IP address was blacklisted on the CBL due to an infection of one of the machines. Apparently CBL detected a trojan on our network, trying to connect to other machines via UDP. So, even though our network was not sending spam, CBL blacklisted the ip address. 

First, PB changed the IP address of outgoing http and connections other than mail, to another ip address which he had available. Then, CBL blacklisted the new ip, with the additional info that no spam was detected, only the presence of the trojan:

IP Address x.y.z.a is listed in the CBL. It appears to be infected with a spam sending trojan, proxy or some other form of botnet.
It was last detected at 2013-10-17 11:00 GMT (+/- 30 minutes), approximately 16 hours, 30 minutes ago.
This IP address is infected with, or is NATting for a machine infected with the ZeroAccess botnet, also known as Sirefef. More information can be found from Wikipedia. It is most often used for bitcoin mining or click fraud, but as it contains a downloader portion, it can do anything.
If this IP address is a NAT gateway, it should be possible to find which computer on your internal network is infected by implementing a filter on your firewall to detect and log attempts to send UDP packets to the Internet with a destination port number of 16471. 

In Cyberoam, System -> Diagnostics -> Connection List tab,  port number of 16471 was seen in the destination section. That machine was cleaned up using Norton Power Eraser as recommended by CBL. Issue resolved for now.

beware of distributed brute-forcing

An interesting article about distributed brute-forcing by bad guys - 

https://bsdly.blogspot.com/2013/10/the-hail-mary-cloud-and-lessons-learned.html

So, for SSH we can use key-based auth for protection, and it would be safer to use two-factor auth for other things like wordpress.

connecting to our ssh server via mobile internet

Due to various power outage issues in the region, I've had to fall back on ConnectBot to connect to our servers. The ip addresses for Airtel 2G seem to be all over the place, as given at https://as.robtex.com/as45609.html#bgp

So, instead of white-listing all the ip ranges 106.0.0.0/8, 223.0.0.0/8, 27.0.0.0/8, 171.0.0.0/8, 182.0.0.0/8, and so on, just added our Voxel server's key to authorized_keys, and added its ip to the allowed range.

Took me an extra ten minutes to do this. First,
host media.radiosai.org
gave me the wrong ip address to allow - the Cloudflare CDN's ip address! Then, the copy-paste of the id_rsa.pub added newlines, so I got errors like error: key_from_blob: can't read
as seen in /var/log/messages of the server. Removed the unnecessary newlines, got it working - there is no need to restart sshd when adding keys. 

saiwaves not booting - manual boot choice needed

With all the power issues we face, we've had to shutdown our servers quite a number of times. saiwaves refused to come up - and the following was PB's solution.

These links helped....

http://en.community.dell.com/support-forums/desktop/f/3514/t/19464480.aspx

http://ubuntuforums.org/archive/index.php/t-2042306.html

use F11 - Boot Menu  

In the boot menu select Hard Drive C: 

then select Slot 2 (Virtual Disk)

Cloudflare optimizations break javascript

Our web-based audio player seemed to have broken all of a sudden. Strange symptoms - working on Safari, but not on Chrome or IE or Firefox. Checking the source, found some Cloudflare injected code. Tried pausing cloudflare, that solved the issue. Then found that Cloudflare optimizations are known to break javascript, jquery and so on. Turned off optimizations, made the settings CDN only, modified the rules for media.radiosai.org also to omit the optimizations, then all is well.

The search box on every page of radiosai.org also had broken due to the javascript issues - removing Cloudflare optimizations solved that issue also. 

saving a video with effects applied via vlc

In my previous post, I failed to save a video with effects. But now, this page showed me how to do it. The gui-based method is:

1. Show all settings (and not Simple) from Tools -> Preferences -> Interface settings
2. In Preferences, under Stream Output -> Sout -> Transcode, choose appropriate filter
3. From main menu, Media -> Convert/Save
4. Add input files, and choose Convert at bottom convert/save drop-down.
5. After doing required conversion, Tools -> Preferences -> Reset Preferences

Unfortunately, this method did not work out of the box with the vlc-based warp player made by Paul Bourke - the filter is not explicitly shown.

Windows solution for warping - avisynth filter

Cross-posted on small_planetarium yahoo group.

Glad to announce yet another Windows solution for warping. This is an implementation as an avisynth plugin called Planetarium, which is listed at

http://www.avisynth.nl/users/vcmohan/

My short review: Takes around 40 sec for video to load into virtualdub using level=7 (documentation is athttp://www.avisynth.nl/users/vcmohan/Planetarium/Planetarium.html )

On my machine, rendering proceeds at nearly 2 fps as against 3 seconds per frame taken by jpgwarper (the older Windows solution at http://www.lochnessproductions.com/software/imgwarper/iww.html )

There is a slight difference in anti-aliasing quality, which I have to explore with different "level" settings.

Where this can be useful: for situations where a pre-rendered video is needed, like playing back from a Blu-ray player instead of a PC and so on. 

( http://groups.yahoo.com/neo/groups/small_planetarium/conversations/messages/2434 )

occasional smb connection issues at studio

Occasionally, the Linux smb share becomes inaccessible, not seen in the list of shares, to some of the Mac users in the Studio.

Solution: When not able to save to the shared folder, do the following:

Open Finder.

Choose Go Menu, Connect to Server.

Type smb://

in the server address box. 

Click connect. When it asks to connect as what user, choose the Guest option. Then you will be able to open the shared folders on that machine.

using connectbot on Android

ConnectBot is an Android SSH client which runs on older versions of Android also. It could import the RSA key pair once I copied it to memory card from my Linux desktop's .ssh directory. In order to use Ctrl, Esc, Func and arrow keys, it uses the trackball.

ESC = press trackball twice

CTRL = press trackball once.

Tab = CTRL + i

Function key - CTRL + number

Arrow keys = move trackball right/left/up/down

vlc preliminary trials - fail

Paul Bourke mentioned that a VLC plugin for warping is under development. Tried out exporting a filtered video using VLC as given at this forum thread,
https://forum.videolan.org/viewtopic.php?t=55449&f=14

Unfortunately, the output doesn't seem to have the filter applied. Code used was
vlc -I dummy stellarium_movie.avi -v --video-filter="wave" --file-caching="20000"  --sout="#transcode{vcodec=mp4v,vb=8192,scale=1,
deinterlace, audio-sync,acodec=mp4a,ab=128, channels=2}:duplicate{dst=standard{access=file,mux=ts,dst=test\output3wave.mpg}}"

I tried --vout-filter also, as given in the forum, but no go. Output video does not have the filter applied.

vlc -I dummy stellarium_movie.avi -v --vout-filter="wave" --file-caching="20000"  --sout="#transcode{vcodec=mp4v,vb=8192,scale=1,
deinterlace,audio-sync,acodec=mp4a,ab=128, channels=2}:duplicate{dst=standard{access=file,mux=ts,dst=test\output3wave.mpg}}"

Edit: Solution is here.

reviving the old Mac as an internet radio

This is going to be a long post.

It all started with the idea of using the old mac as an internet radio. As its screen had become damaged and showed up scratches, and it was underpowered for most tasks, this seemed a good idea. The final solution was to enable booting to console by installing a base install from mini cd of Ubuntu 12.04 and running a script on login. Network connectivity via the built-in ethernet port. Efforts to use a USB wifi adapter were not successful in the sense that the wifi was not reliable, would come up sometimes and not at other times. This post will chronicle the entire story.

Final working solution:

The internal CD drive was not reading. So the easiest way to upgrade to a new distro, booting from CD, could not be used. An external USB CD drive was also not recognized as a CD to boot from, by pressing C at boot time. The netboot procedure for using a Windows machine as TFTP server was used, with the files needed for the netboot being listed at the bottom of this page. For Ubuntu 12.04 Precise Pangolin, the PPC netboot files are at http://ports.ubuntu.com/dists/precise/main/installer-powerpc/current/images/powerpc/netboot/

For doing the netboot, after setting up tftpd with the required files as above, the Mac needs to be booted into open firmware by pressing Alt+Command+o+f at boot, and the command

  boot enet:0

needs to be entered at the Open Firmware prompt. The Ubuntu netboot installer works, but stops at "getting installer components" with a blank screen, and continues after 15 minutes! The whole base install takes an hour or so, downloading required files online took around 15 minutes with our 2 Mbps shared link.

The machine was made to auto-login to the user shell by editing /etc/init/tty1.conf. An interactive bash shell script was the choice for running the "radio" - a simple bash case statement with a while loop asking for input to change streams. Changing streams by killall mpg123 followed by the relevant mpg123 stream-url. This was made to start on boot by putting it in .bash_profile. Then the system boots up and starts playing in 90 seconds.

To prevent sleep when lid closed, used the tip given at http://mrob.com/pub/ibook.html and put a plastic CD case between the lid and the bottom half.

Ease of shut-down - the Ctrl-Al-Delete method given here was adapted, editing /etc/inittab and changing the shutdown -r to shutdown -h to halt the system instead of reboot on Ctrl-Alt-Del. On the Mac keyboard, the keypress for del needs Fn+Delete, since Backspace is labelled as Delete.

Mis-steps:

In hindsight, the final result above could have been accomplished in around 10 minutes' time. Instead, a week was spent in various explorations. Many things were learnt, which are listed below.

Initially, the idea was to upgrade the running Ubuntu 8.04 LTS to 12.04, so that the newer kernel would have built-in support for the USB wifi device. Upgrade from 8.04 to 10.04 went off OK, via

#apt-get update

#apt-get upgrade

#apt-get dist upgrade

#do-upgrade-release

But 10.04 to 12.04 failed.

At this point, I'd tried to get a console-only install by removing all X related stuff. The tasksel method did not work, nor did apt-get remove ubuntu-desktop. But using wildcards and aptitude, removed ~ngnome , ~nxorg and so on, till finally the system booted to a simple bash shell and no X. Getting a list of all installed packages would sometimes be useful.

Perhaps this was because of not doing the
#apt-get update ? Software sources were manually changed archive.ubuntu.com/ubuntu to ports.ubuntu.com , and then doing the above,  do-release-upgrade worked. But at the end of the install, some errors were reported, and on reboot, the system went into a loop and refused to boot. Single user recovery console was the only working option, and ifupdown etc seemed to be borked. So, a fresh install was attempted.

USB disk install as given at https://help.ubuntu.com/community/Installation/FromUSBStick did not work. But then, it was done using the mini.iso, and that page says mini older than 13 does not boot from USB. Then the netboot option mentioned at the top of this post was tried, and worked.

Several times when the Mac was booted without an ethernet connection, the date and time were off. Apparently these iBooks don't have a BIOS battery, and only use a capacitor and the main battery. The capacitor as well as the main battery seem to be dead in this case. Resetting PMU and PRAM etc did not help.

Possibly due to the power constraints of the USB 1.1 port on this iBook, sometimes booting with the DWA-131 USB wifi adapter would cause repeated errors scrolling by every second, complaining unable to enumerate usb device on port 1 - connecting via a USB hub stopped this issue. After the driver was detected, removing the hub also did not cause this problem to recur.

In this case, lsusb showed the appropriate device name, and checking the driver for it, googled 07d1:3303 D-Link System DWA-131 wireless n nano adapter linux and found the driver was
https://wiki.debian.org/rtl819x

also googled 07d1:3303 D-Link System DWA-131 wireless n nano adapter ubuntu and found via
http://ubuntuforums.org/archive/index.php/t-1392153.html

that the command to be used to connect via wifi is iwconfig and not ifconfig.

Also, found that on listing drivers, lsmod was listing the driver rtl819x as loaded, so probably it should work.

So, did
#apt-get install wireless-tools

and tried to connect, as given at this post - in that page, in the "Make it a script" section, perhaps the first line should be ifconfig wlan0 up

Anyway, tried it, did not work. Finally found via this thread that iwconfig works only for WEP, and that wpa-supplicant has to be used for WPA. Tried the methods given here, did not work, so tried to install ubuntu desktop. This was in Ubuntu 13.10 - but starting X was causing a segmentation fault. Even after googling and trying to remove other errors, and following the steps given here, X still seg-faulted when a window manager was installed.

So, went back to 12.04 to try the procedure given at this post to load X - but before doing the X install, thought of trying to connect with commandline using wicd and found this post. This method succeeded! Except in the last line, it was dhclient wlan0 and not dhcpd wlan0 in my case.

# ifconfig wlan0 up  

# iwconfig wlan0 essid "my_network"  

# wpa_supplicant -B -iwlan0 -c/etc/wpa_supplicant.conf  

# dhclient wlan0

Then used this method to make it come up at boot, editing /etc/network/interfaces. But the wifi connection was erratic. Sometimes would not connect at all on boot. Sometimes would connect if the time was set using open firmware with

decimal dev rtc 09 34 13 28 08 2013 set-time

but sometimes not. Would sometimes connect if ethernet cable was connected, sometimes not. Tried going to WEP instead of WPA, and it seemed to work as per the iwconfig method given above, and seemed to connect OK even when the time was off, even when ethernet was off. But a cold reboot the next day showed that the wifi connection was still erratic, and so wired was the way to go.

Testing the networks with wicd, needed to enter wlan0 in Preferences of wicd-curses as per this post.

mpg123 failed to open module  etc errors as non-root user. Installed alsa, rebooted, fixed.
#apt-get install alsa alsa-tools

Then, choosing amixer set Master 45 seems to be a good volume level to set.

After all this, needed to install ntpd, otherwise time is off
https://help.ubuntu.com/10.04/serverguide/NTP.html

blogger post editor broken on IE 8

Probably part of Google's "older browsers not supported" campaign. The post editor seems to be working only in html mode and not in compose mode when using IE 8. Tried clearing cache, still no go. So finally it's time to say goodbye, I suppose, and use Opera or Chrome instead, as a secondary browser. Or I can post like this, using email.

youtube video to mp3

There are quite a few services offering conversion of youtube videos to mp3. vidtomp3.com did a reasonably good job, doing an hour long video in less than 10 minutes. Ad supported, of course. Some other sites have size limits, this one seems to be free of duration restrictions.

DD links and bug in Description form

While adding links to the Descriptions for Divine Discourses found that the form for adding Descriptions which we use internally has some bugs, with erratic behaviour for filenames which have spaces and or other characters. Have alerted PB on this. Meanwhile, the links were added without quotes, since the form again needs special treatment for quotes, and various bash csv export scripts may misbehave with quotes. I was pleasantly surprised to find that even target=_blank works without quotes.

mysql db table repair for phplist

As a follow-up to this earlier post on table repair, PB suggested the following:
He found that the following one line command checks, repairs and optimizes all the databases.

 mysqlcheck -u root -p --auto-repair -c -o --all-databases

This could be used  in situations where we are not sure of which tables have crashed or needs repair.....

easily importing Nokia contacts into other apps with vcf support

Guest post from V:

Create a back up of contacts using application on the Nokia phone. By default it will be saved as Backup000.NBF. The extension looks strange, but the file is similar to an archived file like .zip

Open Backup000.NBF using any archive manager and navigate to /predefhiddenfolder/backup/WIP/32/contacts Extract the folder containing all contacts as .vcf files to another location

Open Terminal and navigate to the folder containing the .vcf files. [Alternate method: sudo apt-get install nautilus-open-terminal adds a right-click “Open Terminal” option into the Nautilus File Manager in Ubuntu.]

Once inside the folder on Terminal, use: cat *.vcf > combined.vcf to combine all .vcf files into a single file.

This file can now be easily imported into Google Contacts or any other application that accepts .vcf files

Edit: There is one more elaborate method of VCF multiple contact import given at http://www.allaboutsymbian.com/forum/showthread.php?p=488142

sharing audio links - sharethis custom link

One of the requests from our listeners was an easy way to share audio links. Exploring the different ways to share, found that sharethis.com does support custom urls and custom text, as given in their documentation. But directly plugging this into the Download.php page would make it a bit ugly, with a separate email for each download link. Something better needs to be designed.

More Seeker bugs

Seeker Theater Edition, Build 828 as per the startup splash screen - The items under Help menu don't seem to be working - Help -> Manual... , Help -> Support... , Help -> About Seeker don't have any effect.

Also, the 'Solar System Map' doesn't seem to be working. When it is enabled, the Sun and planets (if they were visible) disappear. But the map as such does not appear.

Edit: The About Seeker and Solar System Map work when not in "Fulldome" mode, and when the flat screen version of "None" is selected in Settings -> Dome Projection. More info in the post on further experiments with Seeker. 

Software Bisque Theater Suite trials

Trying out the Seeker Theater Edition, found that my machine is not capable enough to do realtime rendering of hi-res - 1920x0180 or more - accelerated video. Frame rates hover around 5 fps. Offline rendering is good, though. Offline rendering of the "Grand Tour" into warped 1920x1080 movies averaged around
5.5 fps for MJPEG / Photo-JPEG codec mov
4 fps for targa frames (slowed down by hard disk)
2 fps for PNG frames (slowed down by compression)
2 fps for H.264 codec mov (slowed down by compression)

and so on.

JPG frame export fails - the files are written as 0 kB files.

Offline rendering of 2K movies averaged
3 fps for MJPEG / Photo-JPEG codec mov

2 fps for targa frames (slowed down by hard disk)
1 fps for PNG frames (slowed down by compression)

Offline rendering of 4K went at around 1 fps for MJPEG / Photo-JPEG codec mov.

Noted that the anti-aliasing option under Settings -> Graphics setup don't seem to affect the offline rendering. The offline rendering seems to be done without anti-aliasing. Sample parts of the 4K frame, zoomed in and cropped,

But the whole frame by itself looks good,

Interestingly, the stars render much smaller in relative size in the 4K frame as compared to a 1K frame, as below:

Unfortunately, there doesn't seem to be any setting option to control the size of the stars. They seem to render the same size whatever be the size of the entire frame, so in a lower resolution frame, the stars are too clunky. So, if the stars look too big, I have to render as large fisheye masters - 3K or 4K - and then warp them down to our 1920x1080 warped files.

Now, all this was about Seeker. The Theater Suite also has Mojo, Syzygy and SkyX Theater Edition. Mojo doesn't seem to do realtime warping - just makes a timeline of multiple video files. Syzygy launches all the components of the Theater Suite in a multi-monitor configuration. SkyX has similar capabilities to Stellarium and accepts map files for warping. Probably I'll be using only Seeker, and doing offline renders.

Edit: More Seeker bugs are reported here, as well as on bisque.com (login required). Also, documentation is lacking on script commands like WAYPOINT2.

bug fixes to audio search

V had come over for Gurupoornima, and kindly helped bug-fix the radiosai audio search results page.

The first bug was displayed when searching for Discourses: if the description is exactly the same, in the search result, only one of the DDs is displayed, even though there are two different versions in different languages. Solution was simple - changed line

$dataQuery.= " group by description ";
to
$dataQuery.= " group by description, language ";

The next bug was harder. If the search result had a very large number of files within a single line, i.e. with the same description, clicking the download button used to fail, or give a truncated list. Checking out the database calls one by one by giving echo statements, V found that the truncation was happening at the group_concat sql command. Apparently mysql has a group_concat default size limit of 1 kB. The blob returned is truncated to 1024 bytes. Googling found the answer. Since we don't have system-wide access on that server, we had to change the size limit for the particular session. Just adding the line

$inc_result= mysql_query("SET @@group_concat_max_len = 512000;");

just before the relevant queries were fired did the trick. The @@ means for this session.

online buying of physical products from overseas

Found out the hard way about Indian customs laws when we ordered the Theater Suite from Software Bisque. The package, sent via UPS, was detained at customs, and the Indian UPS partner asked us for IEC code. Luckily in our case the University has import licence (and duty exemption) so we got the package cleared, even the process took a month. But other institutions may not be so lucky.

Google Reader shutdown - Google player widget no longer up

HMHV blog had all player code hot-linked to the google audio player at
http://www.google.com/reader/ui/3523697345-audio-player.swf

 Google Reader shut down on the 1st of July 2013 and along with it all its allied services. So I was asked for a simple solution. I suggested:

https://sites.google.com/site/mori79/html-gadgets/media-players/google-reader-player

Where it says upload on your own site, you could use sites.google.com or something like that.

The solution preferred was a direct HTML5 embed, like
http://www.w3schools.com/tags/tag_audio.asp

But of course this has mp3 embeds, which did not work on Firefox. But the latest versions of FF on Windows do support this.

HDMI to VGA convertor

V wanted to use the Toshiba AC-100 smartbook for his presentations. Since the projector did not have a HDMI port, he had to convert to VGA. We ordered this product, and tried it out. First testing with the Blu-ray player, did not work. Probably the Blu-ray player does not supply enough power via HDMI. Tried with the AC-100, worked fine. Default output is 1920x1080 @ 60 Hz. Saw the output both with the LG Studioworks 221U monitor as well as with the Optoma TX1080 projector.