avi-to-avi pre-warp tool - GL_warp2Avi

My quest for an OpenGL accelerated pre-warping tool similar to TGAWarp finally made me start coding, albeit in copy-paste fashion. This tool is cobbled together using code from:

http://www.codeproject.com/Articles/8739/Extracting-AVI-Frames - for getting frames from AVI
http://nehe.gamedev.net/tutorial/playing_avi_files_in_opengl/23001/ - for playing AVI with OpenGL
http://paulbourke.net/dataformats/meshwarp/ - for the OpenGL drawing code
http://paulbourke.net/miscellaneous/lenscorrection/#opengl - for the OpenGL initializations
https://github.com/pdbourke/vlc-warp-2.1 - for the ReadMesh code.

Other minor pieces are credited in the comments.

The software can be downloaded from GitHub. The source repository is also available, for all those who would like to help in making improvements.

Usage: Open input fisheye avi file when prompted. Preview of warping is shown on the main window. Choose Menu item Avi Generation - choose output warped avi file when prompted. The render runs at approximately 2 fps on my 1.9 GHz Athlon laptop.

Please note the following points, which could be improved etc. The hard-coded parts are commented as such, // hard coded! hack! and so on.

1. The current version expects the map file EP_xyuv_1920.map which should be a valid map file, in the same directory as the executable. This is hard coded at the moment. Other valid map files could be put in its place, but I have not tested this.
2. The current version uses only AVI for input and output. This could be improved by using FFMPEG as Asaf of Surreyyasoft suggested.
3. The dimensions of the output are currently limited by display size. If display size is larger than 1920x1080, like in my case with a laptop monitor and a secondary monitor at 1920x1200, the initial frame size is set to 1920x1080, and output will be at this resolution. If the program is run on a computer with a smaller display, the frame size reduces correspondingly, and so will the output. This could be improved by using FBO etc. 
4. The progress bar seems to freeze if the application loses focus (if you click on another window etc). But the rendering still goes on without problems.
5. Currently there is no way to pause and resume. Aborting also results in a crash.
6. Currently, instead of giving an error message, the program takes defaults for input and output. Default input is a file called Face2.avi in a Data sub-directory of the executable directory. Default output is in the active directory -  executable directory or input file directory, with the time-stamp as filename.

Output quality seems to be comparable to TGAWarp. Taking sample frames from the excellent show Cosmic Colors from GLPA, which we're currently exhibiting,

Frame generated by GL_warp2Avi:

Similar frame generated by TGAWarp (actually jpgwarper):

Note: 2 GB file limits for input and output files.
Note 2: Workaround using virtualdub as frameserver - then it can use avisynth avs files, which can source image sequences, mp4, mov etc etc. 

changing exe name in a Visual Studio project

This stackoverflow thread discusses this issue of how to change the generated exe file's name. Either change it in Project Properties -> Configuration Properties -> General -> Target Name
or change it in
Configuration Properties -> Linker -> General -> Output file.

procmailrc file issue

For getting the bounces generated by krishna's mailer daemon, which would be sent to noreply@radiosai.org, the postfix install needs to be configured to deliver mail to radiosai.org domain also, and procmailrc has to be configured.

http://hnsws.blogspot.in/2010/11/bounce-processing-for-yahoo-and-new-ip.html

Had done the procmailrc, now have adjusted main.cf

changed
mydestination = radiosai.org, krishna, localhost.localdomain, localhost

to
mydestination = radiosai.org, krishna, localhost.localdomain, localhost, krishna.radiosai.org

But actually that was not the problem - the problem was that .procmailrc was named procmailrc (no dot at beginning). renamed it with the dot, started working.

fixing / changing HELO of mail server

Bounce processing was not proceeding. It complained that php imap module was not installed. Did

apt-get install php5-imap

and service restart apache2

Then error was, cannot open /var/mail/apachemails

Checked
http://hnsws.blogspot.in/2011/04/bounce-processing-cleanup.html

had to do chmod 660 for /var/mail/apachemails

Some mails are being bounced, like rediffmail,
Jan 10 02:19:32 krishna postfix/smtp[28562]: 112D02A0C11: to=, relay=mx.rediffmail.rediff.akadns.net[119.252.147.10]:25, delay=0.27, delays=0.1/0/0.1/0.07, dsn=5.0.0, status=bounced (host mx.rediffmail.rediff.akadns.net[119.252.147.10] said: 553 sorry, invalid HELO hostname (#5.7.1) (in reply to RCPT TO command))

Checking the sai inspires email, looks like the HELO is being sent as krishna instead of FQDN, gmail show original shows

Received: from krishna.radiosai.org (krishna.radiosai.org. [111.93.16.162])
        by mx.google.com

for older mails, while for newer mails it says

Received: from krishna (krishna.radiosai.org. [111.93.16.162])
        by mx.google.com

Fixed by changing in /etc/postfix/main.cf
 myhostname=krishna   to
 myhostname=krishna.radiosai.org
using
https://major.io/2006/12/26/fixing-invalid-helos/

compiling with Visual Studio 2008 Express

Some of the stuff I went through - once again, just a link dump:

fatal error C1083: Cannot open include file: 'afxwin.h'

Apparently this is because MFC is not included with VS Express.

https://www.google.co.in/search?q=creating+mfc+with+visual+studio+2008+express

http://www.codeproject.com/Articles/30439/How-to-compile-MFC-code-in-Visual-C-Express

had to google for ddk, found a link at
http://forums.mydigitallife.info/threads/16561-Looking-for-quot-Windows-Server-2003-SP1-DDK-quot-download-link-needed

Similarly, after building in debug mode, there was the error on running the application that MFC42D.DLL was not found. Since I did have  MFC42.DLL, I just rebuilt it in Release configuration, and the application worked fine. Other resources for these are this MS forum thread and this useful tool.

Initializing window size on starting of application using a call to MoveWindow. Unfortunately, it does not allow the window size to be greater than the size of the screen. So, I can't make AVI using AVI generator demo which are 1920 x 1080, since the window decoration makes the client rect smaller than 1920 x 1080 even if my screen resolution is that big. 

Tried with adding scrollbars, but only the part of the window which is visible is rendered in the avi. Rest is black. 

hardcoded to

createwindow()
cs.style |=   WS_HSCROLL | WS_VSCROLL;

ReSizeGLScene()

hard coded
glViewport(0,0,1920,1080);   

and in setbitmapheader in avigenerator

// changing size of image so dimension are multiple of 4
    //size.cx=(size.cx/4)*4;
    //size.cy=(size.cy/4)*4;
    // hack - hard coding to 1920 x 1080
    size.cx=1920;
    size.cy=1080;

Have stopped working on this for now. Maybe Fluag will work if the developer responds.

compiling Fluag

Tried compiling Fluag directly using the vcproj project file available with the source. Visual Studio 2008 complains that Lua_5.1.5_static.lib is compiled with a different version of the compiler. Tried compiling lua to re-create the lib file - downloaded lua 5.1.5 source, the visual studio solution files from lua-users wiki, but found that the 5.1.5 source contained an etc folder with build instructions using a bat file from Visual Studio terminal, (not simply cmd.exe, we have to go to the Terminal from within Visual Studio, otherwise cl.exe not found errors will come up), used that bat file, it built a lua51.lib, renamed it and moved it to the fluag src/libs folder, ran the build once again, success.

But even with this compiled version, no opengl output for me. 

more caveats with using vlc-warp compiled from source

Following up from my earlier post on compiling vlc-warp on Linux, there were some issues faced by T and posted to the small_planetarium yahoo group. Apparently the warp was not happening, the error message on the terminal was that the mesh file was malformed. Finally, the issue turned out to be locale dependent. Either using a US English locale throughout, or changing . to , in the mesh file, solved the issue. Apparently in the Polish locale, the separator for decimal point is comma , and not dot .

So, while compiling and using vlc-warp, if your locale uses some other decimal mark other than ., you will need to modify the mesh files from Paul suitably!

Fluag for opengl with avisynth

Fluag - downloaded and tried out. The test scripts which do not use opengl work fine. The info opengl script which prints out gl info works fine. But any test script which uses opengl to render video was returning a blank clip. Have requested help from the developer. 

lots of opengl trials

Just copying over all the links I used as such, not bothering to format it:

http://forum.doom9.org/showthread.php?t=145266

points to

http://youka.de/archives/182

http://forums.animesuki.com/showthread.php?t=52757

For the youka.de link,

http://fr.openclassrooms.com/forum/sujet/error-c2440-encore-vc-express-2005-55168?page=1

and
http://stackoverflow.com/questions/5643172/glfw-failed-to-open-a-window

and

http://stackoverflow.com/questions/4845410/error-lnk2019-unresolved-external-symbol-main-referenced-in-function-tmainc

http://stackoverflow.com/questions/6222509/getting-many-opengl-errors

says add windows.h before every include of gl related h file - that worked for fbo test from http://www.songho.ca/opengl/gl_fbo.html

For making a test with Youka's window.h - wait for keypress on Windows,
https://www.google.co.in/search?q=c%2B%2B+wait+for+keypress

http://www.cplusplus.com/forum/beginner/4533/
 

OpenGL in Avisynth

This blog post
http://www.morethantechnical.com/2009/01/14/opengl-for-avisynth/
describes using opengl for an avisynth plugin. Getting the source files from google code,  tried building the dll. Faced several setup issues, but finally got it to compile. But trying it out with Avisynth, found that the plugin was not delivering video out back to Avisynth. It was opening a new Opengl window, but nothing was being drawn to it, and nothing was coming back to Avisynth either. Then I thought of trying out the more advanced Fluag plugin developed by Youka.

installing and trying out SIVP for Scilab on Windows

Exploring more methods of reaching my goal of a graphics-card accelerated way of pre-warping, came across SIVP for Scilab, which seemed to do just what I wanted. That is, it is capable of opening and saving avi files, and can do image manipulations on individual frames with OpenCV acceleration. Tried installing Scilab for Windows, OpenCV for Windows and then the SIVP package listed in the download section on the SIVP page. Opening Scilab and choosing the menu item Toolboxes -> SIVP as given in the documentation pdf, the following errors:

Startup execution:
  loading initial environment
 SIVP - Scilab Image and Video Processing Toolbox 0.5.0
Warning: Feature str2code is obsolete.
Warning: Please use ascii instead.
Warning: This feature will be permanently removed in Scilab 6.0.0
if MSDOS then
     !--error 4
Undefined variable: MSDOS
at line      64 of exec file called by :   
-0.5.0\\loader.sce")
in  execstr instruction    called by : 
execstr(toolboxes(1));if exists("%oldgcbo"
while executing a callback

Also, testing it by calling imread etc gave errors of unrecognized function.

Then tried uninstalling SIVP and installing using ATOMS instead,
atomsinstall('SVIP')

This worked fine. But trying to read images made Scilab give out of memory errors, asking that I use stacksize to increase the stack size. Once again, errors while using
stacksize('max')
stacksize: Cannot allocate memory.

But giving a specific value, like
stacksize(50e6)
worked. Giving larger values, like
stacksize(100e6) 
again failed.

compiling avisynth plugins with opengl

Used Visual Studio 2008 and the instructions at the avisynth wiki. I used the avisynth.h version bundled with the avisynth 2.58 download, by choosing to install Platform SDK also while (re-) installing. For compiling dlls with opengl and glut, followed this very good answer on stackoverflow. Downloading the GLUT libraries and header file, copying to the correct folders, then the settings needed in Visual Studio -

Link to the OpenGL libraries (important step):

• Under the Project menu select Project Properties (Alt+F7) at the bottom.
• Select Configuration Properties → Linker → Input from the navigation panel on the left.
• Select All Configurations from the Configuration drop-down box at the top of the dialog. This ensures you are changing the settings for both the Debug and Release configurations.
• Type “opengl32.lib glu32.lib glut32.lib” in Additional Dependencies and click OK (the opengl32.lib and glu32.lib are already in the system, and glut32.lib will be after downloading GLUT).

It is also necessary to set the paths in Visual Studio:

• In Tools -> Options -> VC++ Directories -> Include Files:
  

  C:\Program Files\Microsoft SDKs\Windows\v6.0A\Include
  

• In Configuration Properties → Linker → Additional Library Directories:
  

  C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib
  

Microsoft account creation email verification problem and solution

In order to compile avisynth plugins, the method with the most step-by-step instructions seems to be Visual Studio, and VS Express is a free download. But for using it for more than 30 days, it needs to be registered, and for that, one needs to create a Microsoft account. In the email verification process for that, unfortunately they don't provide a "copy this code and paste it" form as an alternative to clicking the link which comes in the email. And unfortunately the confirmation link did not work for me - tried it on Firefox, Chrome and IE8. Checking the source of the confirmation email, thought that the length of the URL might be too large, and that that might be the issue. Sure enough, I copy-pasted the URL without the trailing huge

&ru=3Dhttps etc... 

part, thinking that ru must be return url,and sure enough, it worked! So, the URL which worked had the following format,

https://account.live.com/Email/Verify?otc=(92-character-code)&mn==3D(email-in-url-encoded-form)

compiling vlc (or vlc-warp) for Windows

Could not find any relevant solution to the cross-compiling issue - the dlls etc have not been made properly, so even mounting the Linux partition on Windows and trying to run vlc fails. Edit - the cross compiling post has now been updated with the solution - installing wine-dev package.

Tried the Msys method of compiling VLC on Windows. That fails when trying to build automake etc, saying
configure: error: cannot run C compiled programs

Next was to try the "advanced instructions" at  https://wiki.videolan.org/Win32CompileMSYSNew/
That has a note halfway down, saying

Note: I found that compiling 2.1.0 requires xgettext, msgmerge, msgfmt and so on. These in turn require libiconv. I'm not sure if that suffices as at this point I gave up. 

So I also gave up, for now.

cross compiling VLC Warp for windows

VLC wiki recommends cross-compiling from Linux as the preferred route to compile for Windows. But trying to compile the vlc-warp cloned from github, ran into lots of dependency problems. My Kubuntu 12.04 was borked by installing .deb which depended on newer libc versions, so gave it up and installed Ubuntu 13.10. 13.10 initially had older libraries like libc and so on, but after updating, the libraries required were updated to newer ones.

Then, followed the Win32 cross compile guide on the VLC wiki, and the build went through. I followed the "Preferred method" of MingW on Linux. First installed the build tools from the Unix compile wiki,

sudo apt-get install git libtool build-essential pkg-config autoconf

and then did apt-get install for all the packages mentioned on the cross compile guide. I used a tar.gz with a backup of the git clone of the vlc-warp directory which I had got for the Linux compile, and the backup of the vlc-contrib-i686-w64-mingw32-latest.tar.bz2 pre-built contribs which were got earlier while doing make prebuilt for the contribs. The make itself took only around 20 minutes, just like for the Linux make.

Unfortunately, after doing the make,
make package-win-common
failed.

make[3]: *** No rule to make target `/usr/include/wine/windows/stdole2.idl', needed by `stdole2.tlb'.  Stop.
make[3]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc/activex'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/hari/vlc-warp-2.1/win32/npapi-vlc'
make: *** [stamp-npapi-build] Error 2

I guess a solution to this would have to be a separate post.

Edit on 6 Jan 2014: The comments below helped solve this issue. Installing wine-dev package solved it.

clearing up disk space and increasing size of loop drive

My Kubuntu install, upgraded to 12.04 from 10.04, was running out of disk space on root. It was installed on a root.disk file on Windows NTFS partition. First tried googling and found this post, so did

sudo apt-get clean
sudo apt-get autoclean

sudo apt-get autoremove  

sudo apt-get install localepurge

Also tried listing packages with

dpkg-query -W --showformat='${Installed-Size} ${Package}\n' | sort -nr | less

and removing those which I did not need. But still, free space on disk was only less than a GB, compiling was eating up space too. So, tried googling increase size of Linux loop drive, and found this forum thread. That had a lot of "dangerous" warnings. Lost track of it since it was on the Mac, saw the Mac Safari history as given at this post - the History.plist file was a binary file, opened using PsPad's Hex editor.... Then found this solution by googling increase size of /dev/loop0

Booted into Ubuntu 13.10 Live DVD, navigated to the root.disk file.

sudo dd if=/dev/zero bs=1MiB of=root.disk conv=notrunc oflag=append count=1000
(where count is the number of MiB to add)

Then mounted it with
sudo losetup /dev/loop1 root.disk
and resized with
sudo losetup -c /dev/loop1
sudo resize2fs  /dev/loop1

At this point, the system alerts me to
Please run e2fsck -f  /dev/loop1
So I do a
sudo  e2fsck -f  /dev/loop1
and then repeat the
sudo resize2fs  /dev/loop1

Et voila! Done.


Only now did I see the official Wubi guide, which gives a solution to this very problem, in a "safe" way.

screenshots on Linux

Found this page with a review of lots of methods to take screenshots,
http://tips.webdesign10.com/how-to-take-a-screenshot-with-ubuntu-linux

 sleep 10; import -window root MyScreenshot2.png

seems to be the least resource hungry method, since it needs only ImageMagick and not kde etc.

Taking screenshots with vlc, the vlc screen seems to come on top of other windows like options window etc. Or the window ordering for screenshots is different or something like that. Need to arrange open windows in non-overlapping ways to avoid problems.

compiling vlc-warp from git

Paul Bourke has put his vlc-warp on github, tried compiling it for Linux. Before vlc-warp, wanted to check if vlc 2.1 would compile. Downloaded from videolan.org and started following the instructions. (Initially, I tried without reading the instructions, quickly ran into dependency issues like the one discussed here.)

Tried it first on my dual boot Kubuntu 10.04. But trying to get the dependencies using
sudo apt-get build-dep vlc
did not go through - the package versions of the dependencies in 10.04 were too old. So, decided to try it using a Live DVD, Ubuntu Studio 13.10 was available with a colleague with his OSFY magazine subscription.

Initially, tried with the vlc source downloaded onto a Windows FAT32 drive. There, ./configure was not executable - apparently, the drive was mounted noexec or something like that. So, copied the source to the home folder, and started again.

Next error was, "Unable to find a source package for vlc" when trying the
sudo apt-get build-dep vlc
For this, selected both the "source files" and "universe" repositories in the package manager, did
apt-get update, then tried
sudo apt-get build-dep vlc
again, worked now.

./configure took a couple of minutes to run, and make took around 20 minutes on my machine, with an AMD Athlon dual-core 1.9 GHz processor. At the very end, make gave a couple of fatal errors, saying make-alias did not work. Most probably this was due to copying over the source from the Windows drive, which made all the files including make-alias and configure lose their executable permissions. But the only issue was that there was no vlc binary in the root folder, I had to go to the bin folder and choose the vlc-static binary. That worked. And I could choose the OpenGL video output - that is critical for vlc-warp to work.

Then tried with the actual vlc-warp. First tried downloading the source as a zip file using the link given on github. That did not work - make gave errors saying no git repository found or something like that. Then did it properly using
git clone https://github.com/pdbourke/vlc-warp-2.1.git
as given in the VLC build instructions. Then,
./bootstrap
./configure
make
worked fine, twenty minutes later, I had the binary. Tried it out with a standard mesh file from Paul, performance was similar to the Windows version.

In summary: first check with VLC 2.1 installed on your machine if OpenGL video output is available, and if it works. Only if that works will vlc-warp 2.1 work. If your machine does not have a supported graphics card, the OpenGL option either won't be available, or using that option won't get you video using vlc. In that case, vlc-warp won't work either.

Edit: More caveats!

phplist unable to recognize bounce if email id has space before it

Looking at the bounce messages on phplist, found a couple of bounces every day, for which phplist could not identify the list subscriber - listed as Unknown. Viewing the bounce, found that the email id had a space in front of it, which was the reason why phplist could not identify it. Unsubscribed manually. (This is phplist version 2.10.10).

previewing video in Reaper, workaround for AAC

My version of Reaper, 3.52, had problems displaying videos using the FFMPEG option. Tried downloading various versions of the ffmpeg dlls and put them in the Program Files\REAPER folder with no results:

Finally deselected the FFMPEG option, choosing to preview video only with DirectShow - that works fine for me now. Options -> Preferences -> Media -> Video/REX/Misc, Disable FFmpeg decoding.

Most mp4 files have AAC audio. Unfortunately, Reaper (at least my version) seems to have issues with AAC, audio stutters and finally reduces to a single repeated hum after a few seconds. Workaround was to extract audio using Avidemux, save it separately using Audio -> Save Audio - and use that audio in Reaper.

file permissions

Just a note about file permissions with SMB on Linux. P had moved some folders to another folder. As a standard user with ssh, I couldn't delete them - had to be root. P then demonstrated that the folder with files in a Samba share was with owner nobody:nobody so that it could be deleted by users accessing the Samba share as guest on Win or Mac.

quiz code correction

A found a bug in the 9th question of a quiz featured as a Sai Inspires link, which had been uploaded to our website in 2009! The answer was not being displayed. I checked for the reason in javascript, did not find any problem. P found the reason - the "answer area" div had been omitted in the html!

network congestion and the nuclear option

Before starting the evening live broadcast today, listening to AsiaStream took a long time to buffer. Checking cyberoam usage logs, found that WAN download was maxing out. Even the firewall rule was not helping, probably because the connections were made before 4.15 pm, and the firewall rule after 4.15 pm would affect only new connections.

Under these conditions, BUTT would not connect easily. Earlier, I would just restart cyberoam (soft restart) and that would kill all concurrent connections, allowing BUTT to connect.

This time, went into Cyberoam -> System -> Diagnostics -> Active Connections list, and killed around 10-15 of the around 400 active connections, which had used max download bandwidth. Most of them were port 443 - https - most probably youtube. One or two were chat ports 5222 etc. Then was able to start streaming without issues.

ftp speed bump - restart solved

Our internal ftp server which also hosts our mailing lists suddenly slowed down: transfers which went at 100 Mbps over the LAN started going at 3-4 Mbps. A restart of the machine solved the issue. But during the restart, P reports, the system required a manual execution of fsck. Once that was done, system booted. It was our other internal server which needed the manual choice of boot device.

email for a new domain - options

Copy-pasting an email I sent about adding email for a new domain:

I see that the mail exchanger (MX) records are not set for this domain. So, currently, no emails for this domain are configured.

There are many options for email.

1. use the built-in free nettigritty email, or do a mail forwarding to some other account. This may be easiest to configure, but I have not tested nettigritty's mail servers reliability.

2. Use paid mail hosting, like google apps for business which starts from Rs. 150 per user per month. There are, of course, many other options here.

3. Unfortunately google's free plans no longer allow new sign-ups, so the option we chose for our earlier domains is not available. But Windows Live for domains is available for free. We could use that, and if necessary, forward to a gmail / sssbpt.org id also.

4. There is probably one more option - domain alias. Google apps allows you to have aliases, so that email to webmaster@ournewdomain.com can be directed to webmaster@ourolddomain.org etc. 

youtube download failure and solution

Yesterday, for the first time, the youtube mp4 file I downloaded using Flashgot Firefox add-on did not work. Then, another file I downloaded also appeared to be "corrupted". Googling got me this page which indicated that the problem was that youtube was using MPEG-DASH streaming, and so the files were not downloaded correctly.

Tried using Youtube Center as indicated in this Dave's tip, but even with the Youtube Center firefox add-on, when downloaded with flashgot, the downloaded file still had issues.

Then tried the old faithful - the KeepVid.com bookmarklet. Finally, video downloaded with that worked fine. Of course, a caution for keepvid.com users, you have to click on the download link on the right of the text box and not on the bigger "Free Download" link below, which is actually an ad for another downloader product. And keepvid needs to run a java applet.

What could I do with the "corrupted" downloaded videos? This post indicated that GPAC might be able to help. (BTW, Nirsoft's freeware tools are really nice!)

And indeed, the mp4box utility contained in the GPAC download did the trick - made the video readable by AVIDemux using

MP4Box -add file.mp4 out_file.mp4

Even though the documentation said that the -isma switch would make the video compatible with most players,  doing
MP4Box -isma file.mp4

had a similar result as the -add above, with the file being readable by VLC and AVIDemux, but not directshow. Used Avidemux to re-encode the file as HuffYUV, then directshow could read it. Using Avidemux in copy mode was not enough. And after all this, there was no audio stream. Apparently, the downloaded file contained only the video stream and not the audio.

So, the best option would be to re-download using Keepvid or something like that, unless you need only the images and not the sound, like in my case.

Edit: Just noting that mkv files produced in "Copy" mode using Avidemux could be played by TheKMplayer, VLC etc, but not directshow/Windows Media Player. avi files produced in the same way could not be played by TheKMPlayer either, though VLC could.

side-chaining - ducking in Reaper

Following the method given at AskAudio:

1. Drag and drop from the io button of the track which is going to be the controlling track, to the track which is the one which will be affected by the compressor.
2. Choose drop-down and change the destination of the receive to 3-4 instead of the default 1-2.
   
   
   
   
   
3. Choose the Detector Input on the compressor to be Auxiliary Input L+R and other settings as per your choice.
   
   
   

shortcut to creating multiple videos on dome

I wanted to have a number of videos tiled on the dome, maybe six or nine videos in the front upper surface of our tilted dome. The classical method would have been to use blender and a video texture. But that is very time consuming, and my blender skills are extremely rusty. Looking around for shortcuts, the General quadrilateral transform in Virtualdub was the first thing I thought of. But that filter does not support overlays - background remains a single colour. The Reform and Reformer plugins for avisynth did support overlay, but..... Reformer turned out to be buggy, the background video kept moving forward and back and turning green etc.... and Reform's output was too jaggy and aliased.

So, used the General Quadrilateral transform in Vdub, created a mask for each video, and used simple overlay calls in an avisynth script to overlay all the videos.

lrmask=Imagesource("lower_right_white0.jpg",end=450)
lr=avisource("lower_right.avi")

and so on, where lower_right_white0.jpg looked like 

bg=overlay(bg,ul,mask=ulmask)
bg=overlay(bg,ll,mask=llmask)
bg=overlay(bg,um,mask=ummask)
bg=overlay(bg,ur,mask=urmask)
overlay(bg,lr,mask=lrmask)

For getting the shapes right, first I did a trial and error using the warped mesh as a template. But that was not getting me the shapes so easily - the warped mesh has latitudinal and longitudinal lines, not rectangular lines. Not so suitable for placing rectangular videos. Finally, thought of directly painting with gimp on the dome - put the mesh as a background layer for aligning the frame to the dome approximately, then painted the desired rectangles on a new layer. Using those rough outlines, noted the vertex co-ordinates as below.

lower middle video = 440x248 - 1.774
lbotx= 740
ltopx= 740
lboty=1079 #default
rboty=1079 #default
rtopx=1180
rbotx=1180
rtopy= 832
ltopy= 832

upper middle video = 340x220 - 1.54
lbotx= 800
ltopx= 800
lboty= 740  
rboty= 740  
rtopx=1140
rbotx=1140
rtopy= 520
ltopy= 520

lower right video 
lbotx=1264
ltopx=1264
lboty=1079 #default  
rboty=1079 #default  
rtopx=1560 #changed from 1600
rbotx=1600
rtopy= 780
ltopy= 832

calculated lower left
rbotx= 656
rtopx= 656
rboty=1079 #default  
lboty=1079 #default  
ltopx= 360 #changed from 320
lbotx= 320
ltopy= 780
rtopy= 832

upper right video
lbotx=1260
ltopx=1220
lboty= 770  
rboty= 730  
rtopx=1480
rbotx=1550 #changed from 1590
rtopy= 530
ltopy= 600

calculated upper left
rbotx= 660
rtopx= 700
rboty= 770  
lboty= 730  
ltopx= 440
lbotx= 370 #changed from 330
ltopy= 530
rtopy= 600

updating remote database with records in one field

We had to update the 'firstplayed' field of the schedule page database on our website. Our local database had the info. The way to do this seemed to be:

• Export the filename/firstplayed fields from remote db as csv, 
• import into local db as a table, create a view (run a join query) to get the desired firstplayed field for each filename, 
• export as csv, 
• import into remote db, 
• run an update query there. 

But Murphy's law was evident at every step. My database guru roommate Nz helped out to navigate the Murphy minefield. Remote db was MySql, with phpMyAdmin front-end. Local db was Postgres, with phpPgAdmin front end. The import button comes up in phpPgAdmin while viewing a particular table.

• phpMyAdmin's default csv output had ; (semicolon) as default separator, phpPgAdmin's import needed , (comma) as separator. 
• The remote db often took a long time to respond since it is on shared hosting. 131 seconds to update 2000 records, for example.
• phpPgAdmin demanded the field names as the first row of the csv
• phpMyAdmin ignored the field names as the first row, took that row as data to be entered, and mixed up the fields - firstplayed was entered as filename and vice versa!
• Since there were many streams, the firstplayed from the local db had multiple matches, quite a few with NULL. Just ignored them, and used a NOT NULL condition. Syntax for postgres - IS NOT NULL - not not equal NULL. :)
• Had to do a left() firstplayed field, since the local db had a full timestamp, while remote db needed only the date. 
• Telugustream has a lot of files yet to be updated in the temp_play_history table. So will have to do the above procedure again.
• Finally, the update query syntax for mysql was non-standard - Access style, as my db guru roommate mentioned - instead of update dt set df=sf from dt,st where d.jf=s.jf, the syntax was
  update `dt`,`st` set `dt`.`df`  = left(`st`.`sf`,10) where `dt`.`df` = `st`.`sf`

terminal credentials in the new internap portal

After Internap took over Voxel, the configuration portal was "upgraded". In the new portal, the ssh terminal credentials are shown when  you click on your.servername.com, scroll down to device access, and click on the show link next to the ssh password. The confusing character in the middle of our password is capital i (which might have been small L or the number one).

Sureyyasoft Shira Player mini review update

In my earlier post about the Shiraplayer, I had mentioned the warp quality issues it carried forward from its stellarium base. Now the developer has added a better warp functionality in the latest release 1.5.1 - a custom warp data file module.

Initially, on using this version, I got blockiness, as in the screenshot below.

The developer suggested I restart the application after choosing the warp file, and try again, this time it worked fine. So, no more warp quality issues.

Unfortunately my PC (Athlon 1.9 GHz laptop) is not fast enough for realtime usage of shiraplayer with warping.

The Shiraplayer process with the new warping mode takes approximately as much cpu as the warping vlc player produced by Paul. But Shiraplayer has two processes running, one with the video and one with the other features. The process with the other features takes up nearly 50% cpu on my system. So, Shiraplayer stutters while playing even those videos which play OK on Paul's realtime player. For eg, even the 1.2K videos which take up 30% cpu on Paul's player, playing back at full 30 fps, are played back only at 15 fps on Shiraplayer.

But probably all these will not be an issue for those with more powerful PCs. Probably a decently specced machine from the last two years should be able to handle it just fine.

video editing with After Effects

Since the blender methods for cross-fades were quite cumbersome, tried out After Effects instead. What I have is AE 4.1. For fades, need to set opacity keyframes for the upper track. For previewing audio, have to scrub the timeline with Ctrl pressed. Audio doesn't play when the time controls are used. Menu items Composition -> Preview -> RAM preview or Audio preview can make the audio play. Will try this, check if this is better than installing my old copy of Adobe Premiere. 

video editing with blender

Following the steps given at this post. Basically, choose Video Editing in the Choose Screen Layout drop-down just to the right of the menus, and change the default Graph editor window into a Properties windows when you want to do the export.

Right-click and drag to move clips. K to cut. Shift Right-click on two clips to select both of them, then choose bottom menu item Add -> Effect Strip... -> Gamma Cross to cross fade. Have to enable AV sync if needed, default is no sync.

And as this tip says, to move a view-port to a different monitor, shift+left click on the upper right corner of the view port to make it a separate window.

To get rid of a view port, as mentioned here, move the cursor near the border with the next area till it becomes a double-sided arrow, right-click and choose join areas.

The scroll bar handles can be used to expand/contract the timeline view in the Video Sequence editor.

Mouse scroll wheel also zooms in and out. Middle mouse button to pan. Full details are at the Blender wiki.

Select a clip and hit k to cut. Right-click and drag on the arrows at either end of clip to trim.

another audio cleanup tool-chain - Reaper for noise reduction

In previous posts, I've mentioned using Cool Edit Pro and Sound Forge for audio cleanup. For audio with a lot of hiss, noise reduction with Cool Edit Pro produces a lot of artifacts. Reaper's ReaFir filter in Subtract mode is useful in such situations. For an old recording taken from a VHS tape, the following steps were used.

NR and gating in Reaper,

Then sweetening the sound using the EQ setting "30-band Punch and Sparkle" in Cool Edit Pro,

and finally RMS normalizing to -10 dB RMS with Sound Forge, choosing regions of roughly equal loudness.

Edit: 13 Feb 2020 - adding a couple of videos to demonstrate choosing regions of equal loudness by visual inspection.

and then normalizing them with keyboard shortcut Ctrl Y to speed up the process.  

taking screenshots on Mac OS X

This wikihow article has seven methods of taking Mac OS X screenshots.

Command-Shift-4  -> screenshot region of interest, saved on desktop

Command-Shift-3  -> grab full screen, saved on desktop

Command-Shift-4-Spacebar  -> grab an open window, by clicking on it, saved on desktop.

Adding Ctrl like Command-Ctrl-Shift-3 etc copies to clipboard instead.

Applications -> Utilities -> Grab application is also available. Some extra tools are also mentioned.

more details of Reaper-BUTT broadcast setup

My post on the setup for live broadcasts using Jack OS X was a bit skimpy on the Reaper setup details. These screenshots below tell the story of the setup on the Reaper side.

I've got sends going to a full-mix recording track as well as the broadcast track, so that I can use them without interruption even when I 'solo' any of the tracks to listen to that mic - the 'solo' happens by muting all other tracks in the mix which goes to the headphone (outputs 1-2). The mix which is recorded goes to outputs 3-4. I also have another Jack setup where output 4 gets a compressed signal from the broadcast channel below - which could be sent to the video team, for example.

On this broadcast track, I have a high pass filter to cut out rumble, knee at 100 Hz, an aggressive compressor, and the insert from Jack. The insert's return is used as input for BUTT using Jack routing.

VLC-based Warp-on-the-fly player from Paul Bourke - mini review

Paul has now released the VLC-based Warp-on-the-fly player. Here was my mini-review for the beta test done on Oct 4.

As Paul mentions, there are some performance issues. Relatively low bitrate / lower resolution files are OK. The sample1.mov file  - 1280x1280, 6.4 Mbps with avc1 codec - plays smoothly. A 2K file I encoded with Xvid codec, 9.8 Mbps, plays at something like 15 fps, with more frames being dropped where the scenes are more complex. Seems to be CPU limited in my case, so people with beefier CPUs should be OK. Mine is an Athlon 64 X2 dual core processor at 1.9 GHz, with Nvidia Geforce 9100 M Graphics.

CPU usage -

1280x1280, 6.4 Mbps with avc1 codec with warp - 55% with beginning peak of 75%

1280x1280, 6.4 Mbps with avc1 codec without warp - 40%

2048 x 2048 Xvid codec, 9.8 Mbps with warp - hits 99, 100%, hence frames dropped to 15 fps or so.

2048 x 2048 Xvid codec, 9.8 Mbps without warp - 65%, with peaks of 75% for complex scenes.

Another issue for me is that the snapshot feature, or the convert feature, does not capture the warped file, but instead captures the input frame. Perhaps there is some setting which I should do in order to capture / export the warped frames from this player? If I can get the capture / export working, then the performance issues will be moot......

Sureyyasoft Shira Player mini review

Tried out the Shira Player from Sureyyasoft - a player modified from a Stellarium base. At first it crashed, the developer sent me a shader file to put in the data folder, after which it works. Interestingly, it does on the fly warping of fisheye movies using the spheric mirror distortion built into Stellarium. But as mentioned in my previous post on Stellarium, the output image quality when warped is not so great, especially when I use image capture and compare with the other warping tools.

Image obtained from the avisynth plugin, using Flipvertical():

Image obtained from Shiraplayer:

Additionally, there seems to be some sort of up-down flipping involved: Flipvertical() was used in the avisynth script to get a similar image.

Update: Newer version of Shiraplayer with these issues solved, discussed here.

infection on Studio network

Our IP address was blacklisted on the CBL due to an infection of one of the machines. Apparently CBL detected a trojan on our network, trying to connect to other machines via UDP. So, even though our network was not sending spam, CBL blacklisted the ip address. 

First, PB changed the IP address of outgoing http and connections other than mail, to another ip address which he had available. Then, CBL blacklisted the new ip, with the additional info that no spam was detected, only the presence of the trojan:

IP Address x.y.z.a is listed in the CBL. It appears to be infected with a spam sending trojan, proxy or some other form of botnet.
It was last detected at 2013-10-17 11:00 GMT (+/- 30 minutes), approximately 16 hours, 30 minutes ago.
This IP address is infected with, or is NATting for a machine infected with the ZeroAccess botnet, also known as Sirefef. More information can be found from Wikipedia. It is most often used for bitcoin mining or click fraud, but as it contains a downloader portion, it can do anything.
If this IP address is a NAT gateway, it should be possible to find which computer on your internal network is infected by implementing a filter on your firewall to detect and log attempts to send UDP packets to the Internet with a destination port number of 16471. 

In Cyberoam, System -> Diagnostics -> Connection List tab,  port number of 16471 was seen in the destination section. That machine was cleaned up using Norton Power Eraser as recommended by CBL. Issue resolved for now.

beware of distributed brute-forcing

An interesting article about distributed brute-forcing by bad guys - 

https://bsdly.blogspot.com/2013/10/the-hail-mary-cloud-and-lessons-learned.html

So, for SSH we can use key-based auth for protection, and it would be safer to use two-factor auth for other things like wordpress.

connecting to our ssh server via mobile internet

Due to various power outage issues in the region, I've had to fall back on ConnectBot to connect to our servers. The ip addresses for Airtel 2G seem to be all over the place, as given at https://as.robtex.com/as45609.html#bgp

So, instead of white-listing all the ip ranges 106.0.0.0/8, 223.0.0.0/8, 27.0.0.0/8, 171.0.0.0/8, 182.0.0.0/8, and so on, just added our Voxel server's key to authorized_keys, and added its ip to the allowed range.

Took me an extra ten minutes to do this. First,
host media.radiosai.org
gave me the wrong ip address to allow - the Cloudflare CDN's ip address! Then, the copy-paste of the id_rsa.pub added newlines, so I got errors like error: key_from_blob: can't read
as seen in /var/log/messages of the server. Removed the unnecessary newlines, got it working - there is no need to restart sshd when adding keys. 

saiwaves not booting - manual boot choice needed

With all the power issues we face, we've had to shutdown our servers quite a number of times. saiwaves refused to come up - and the following was PB's solution.

These links helped....

http://en.community.dell.com/support-forums/desktop/f/3514/t/19464480.aspx

http://ubuntuforums.org/archive/index.php/t-2042306.html

use F11 - Boot Menu  

In the boot menu select Hard Drive C: 

then select Slot 2 (Virtual Disk)

Cloudflare optimizations break javascript

Our web-based audio player seemed to have broken all of a sudden. Strange symptoms - working on Safari, but not on Chrome or IE or Firefox. Checking the source, found some Cloudflare injected code. Tried pausing cloudflare, that solved the issue. Then found that Cloudflare optimizations are known to break javascript, jquery and so on. Turned off optimizations, made the settings CDN only, modified the rules for media.radiosai.org also to omit the optimizations, then all is well.

The search box on every page of radiosai.org also had broken due to the javascript issues - removing Cloudflare optimizations solved that issue also. 

saving a video with effects applied via vlc

In my previous post, I failed to save a video with effects. But now, this page showed me how to do it. The gui-based method is:

1. Show all settings (and not Simple) from Tools -> Preferences -> Interface settings
2. In Preferences, under Stream Output -> Sout -> Transcode, choose appropriate filter
3. From main menu, Media -> Convert/Save
4. Add input files, and choose Convert at bottom convert/save drop-down.
5. After doing required conversion, Tools -> Preferences -> Reset Preferences

Unfortunately, this method did not work out of the box with the vlc-based warp player made by Paul Bourke - the filter is not explicitly shown.

Windows solution for warping - avisynth filter

Cross-posted on small_planetarium yahoo group.

Glad to announce yet another Windows solution for warping. This is an implementation as an avisynth plugin called Planetarium, which is listed at

http://www.avisynth.nl/users/vcmohan/

My short review: Takes around 40 sec for video to load into virtualdub using level=7 (documentation is athttp://www.avisynth.nl/users/vcmohan/Planetarium/Planetarium.html )

On my machine, rendering proceeds at nearly 2 fps as against 3 seconds per frame taken by jpgwarper (the older Windows solution at http://www.lochnessproductions.com/software/imgwarper/iww.html )

There is a slight difference in anti-aliasing quality, which I have to explore with different "level" settings.

Where this can be useful: for situations where a pre-rendered video is needed, like playing back from a Blu-ray player instead of a PC and so on. 

( http://groups.yahoo.com/neo/groups/small_planetarium/conversations/messages/2434 )

occasional smb connection issues at studio

Occasionally, the Linux smb share becomes inaccessible, not seen in the list of shares, to some of the Mac users in the Studio.

Solution: When not able to save to the shared folder, do the following:

Open Finder.

Choose Go Menu, Connect to Server.

Type smb://

in the server address box. 

Click connect. When it asks to connect as what user, choose the Guest option. Then you will be able to open the shared folders on that machine.

using connectbot on Android

ConnectBot is an Android SSH client which runs on older versions of Android also. It could import the RSA key pair once I copied it to memory card from my Linux desktop's .ssh directory. In order to use Ctrl, Esc, Func and arrow keys, it uses the trackball.

ESC = press trackball twice

CTRL = press trackball once.

Tab = CTRL + i

Function key - CTRL + number

Arrow keys = move trackball right/left/up/down

vlc preliminary trials - fail

Paul Bourke mentioned that a VLC plugin for warping is under development. Tried out exporting a filtered video using VLC as given at this forum thread,
https://forum.videolan.org/viewtopic.php?t=55449&f=14

Unfortunately, the output doesn't seem to have the filter applied. Code used was
vlc -I dummy stellarium_movie.avi -v --video-filter="wave" --file-caching="20000"  --sout="#transcode{vcodec=mp4v,vb=8192,scale=1,
deinterlace, audio-sync,acodec=mp4a,ab=128, channels=2}:duplicate{dst=standard{access=file,mux=ts,dst=test\output3wave.mpg}}"

I tried --vout-filter also, as given in the forum, but no go. Output video does not have the filter applied.

vlc -I dummy stellarium_movie.avi -v --vout-filter="wave" --file-caching="20000"  --sout="#transcode{vcodec=mp4v,vb=8192,scale=1,
deinterlace,audio-sync,acodec=mp4a,ab=128, channels=2}:duplicate{dst=standard{access=file,mux=ts,dst=test\output3wave.mpg}}"

Edit: Solution is here.