new streaming server live

Changed TTL for our streaming server to 10 minutes, and some time later, changed the DNS A record. Tweeted that users should clear their browser caches in case of problems. After half an hour or so, many users were still connected to the old server. Manually kicked them, then many started connecting to the new server. After one day, only a handful were on the old server, so shut down icecast.

changes to our icecast setup

Continuing my earlier post about changes while migrating, we made some changes to the icecast setup - using the on-connect tag to run our live begin and end scripts. There were some small hiccups while doing this - since the she-bang #!/bin/bash was not present on our scripts, they were not being executed. PB corrected this, and then giving the full path in the on-connect directive does the trick.

Also, instead of changing the alias of port 8020 to a different mount-point as earlier, we've now made the fallback-mount change in the mount-point telugustream during the live session.

(Our attempts to avoid the live begin and end scripts did not work. The fallback-mount directive for relay does not seem to work. So, if livemandirasia is a relay for livemandir, and livemandir does not have a fallback, then if asiastream falls back to livemandirasia, then if livemandir is not live, listeners get a 404 file not found. And if connect when live and live disconnects, listeners are kicked.)

In the web player at our website, the streams were linked to server1.radiosai.org:8000 etc - changed to stream.radiosai.org:8000 etc. and purged the cloudflare cache for those files. Other places have stream.radiosai.net, which is fine since it is a cname for stream.radiosai.org.

Then, PB found a nice way to make the icecast daemon start as our user (and not root) on reboot - the @reboot directive in crontab. So, now no manual intervention is required for starting the streams. Also, when done this way, the ulimit workaround we had to implement is automatically taken care of - the cron is run as root su'ed as our user.

key-based rsa auth - a caution

When doing key-based auth for ssh, a caveat - if the .ssh/authorized_keys or .ssh/id_rsa files are being modified, the permissions need to be set to not allow read or write by any other user for the authentication to work - chmod 600 needs to be done. 

changing servers with an upgrade

Migrating data from Internap (formerly Voxel) to Codero, we found that the upgrade from Ubuntu 10.04 to 14.04 breaks many of our configuration files. Apache and vsftpd in particular. Icecast remains at the same version, but our icecast scripts which use small binary files needed re-compiling, of course.

Apache:

• Merely creating symlinks did not seem to work - we had to run a2enmod and a2ensite for them to work properly. This included a2enmod cgi , without which the awstats perl scripts were being shown as plain text :)
• The default website name has changed from default to 000-default - without that, requests to the server using the ip address or using other names will not work. 
• The configuration files in sites-available needed to have .conf at the end.
• The old method of
  Order Allow, Deny
  Allow from all
  had to be changed as per this post to
  Require all granted
• Many more changes in apache2.conf, so we just modified the newer apache2.conf step-by-step with our old requirements. 

Vsftpd:

This needed additional config lines as per this post, done by PB, added the lines 

chroot_local_user=YES

allow_writeable_chroot=YES

and restarted vsftpd to fix the problem with not being able to write using ftp. 

Icecast:

We created symlinks to mp3info, mp3splt etc since the paths to these given in the old scripts had changed. Also recompiled the "CCodes" files. cc filename.c -o filename_of_executable

We tried to run icecast from init scripts, but just changing the USERID and GROUPID in the icecast2 init script was not enough. Perhaps we have to add chroot and run as user parameters also in the xml config file - we have to explore this later. 

We also tried some techniques to try and get Telugustream to mirror the live from mandir stream only during live times, but the methods we tried did not work.

livefrommandirfortelugustream as an alias for livemandir, and having autotelugu as fallback-mount for livefrommandirfortelugustream did not work in that the alias does not work in cascaded fashion - we already have :8020/ aliased to telugustream, but this does not follow the alias to livemandir. 

Also, trying to make livefrommandirfortelugustream a relay of livemandir works, but when livemandir ends, it has the fallback-mount of autoasia, so livefrommandirfortelugustream continues to get the relay audio instead of falling back to autotelugu. 

So we have to retain our method of switching config files using a shell script and manually dumping telugustream users to the relevant mountpoint using icecast's /admin/moveclients.xsl called using wget. 

video setup troubleshooting

The video team had a couple of problems. After changing video switchers from their older Datavideo DV switcher to the newer Panasonic AG-HMX100 HD mixer, they were not able to get a composite analog video out, or IEEE 1394 (firewire) DV out for streaming, and had rigged up a camera facing a screen for getting a DV feed. Apparently the Aja Ki Pro had earlier given them a composite out, but now was not giving it. Checked, found the manual mentions that the component out has to be set to SD in order for the composite out to be available. V did it, and it worked. That composite out is given to a DVCAM tape deck, which in turn gives a firewire out to the streaming server.

The next issue was regarding sourcing of PC playback video to the video mixer. Their setup had VGA out from PC going to a video doubler connected to the display screens in the hall, and a HDMI out from the PC going to a screen with a HDMI -> DVI cable. It so happened that the HD video switcher had a DVI input, only the resolutions supported were 1920x1080, 1280x768 etc. After changing the resolution to 1280x768 (1080p not supported by graphics card) and choosing the same format 1280x768 @ 60 Hz in the mixer settings, it could recognize the PC input as one of the valid sources. 

instapaper epubs on kobo reader

NNR gave me his Kobo Vox ereader to use. The epubs available for download from instapaper do not open in the Kobo's built-in library, but open in FBReader which I installed. Converting a mobi downloaded from instapaper to epub using Calibre could make that file recognizable by Kobo's library app. But with any method, the Feynman lectures' equations don't render correctly in epub. Printing to pdf using PDF995 and reading with Adobe Reader works nicely.  

documenting my fulldome process

Answer to a query about how we make movies from domemasters (still images of single frames):

To make the images into a movie, I usually do my own technical work. My workflow is on Windows. So, I generally follow this method.

Software used:

Avisynth - http://avisynth.nl/index.php/Main_Page

Virtualdub - http://www.virtualdub.org/

After installing the above software, I create an avisynth file, which is just a text file with .avs extension.

Opening this file in virtualdub, I choose
 Video -> Compression Xvid codec, maximum bitrate 9000 (kbps or so)
Audio -> Audio from other file -> choose the audio file
and then File -> Save as avi

Edit 19 Oct 2015: Adding a picture:

data analysis with ROOT - redux

In my previous post about data analysis with ROOT, I had used the data from a single gamma. Extending it to multiple gammas - 1500 - and calculating the energy weighted position, found a bug in my code.

Apparently the Fill() function does not fill if the indices passed to it are 0. That is,
h2->Fill(i,j,energy2D[i][j])
will not work for i=0 or j=0. Since our energy2D C array has a range 0-7 for i & j, this Fill() had to be re-written as
h2->Fill(i,j,energy2D[i-1][j-1])
with the i & j range 1-8.

I also changed my definition of the histogram to
TH2I *h2 = new TH2I("h2","energy distribution",8,1,9, 8,1,9);
instead of
TH2I *h2 = new TH2I("h2","energy distribution",8,0,8, 8,0,8);

With the resulting root file, energy-weighted position analysis could be done quite quickly, almost instantaneously, with

// loading the data in root file
    TFile f("multifile.root");
    f.ls();
    TH2I* h = (TH2I*)f.Get("h2"); 

for (int i=1;i<9;i++)
    {     
    for (int j=1;j<9;j++)
      {
      bincontent=h->GetBinContent(i,j);
      ewposx += posx[i-1][j-1] * bincontent ;
      ewposy += posy[i-1][j-1] * bincontent ;
      totalcount +=  bincontent ;      
      }
    }
    ewposx /= totalcount;
    ewposy /= totalcount;
    printf("ewposx & y   = %f, %f\n", ewposx  , ewposy  );

Changing the position of the gamma was fairly straightforward in LXePrimaryGeneratorAction.cc

The calculation of energy weighted position in the LXe example seems to have broken somewhere in development - people have reported non-zero positions in 2010 and so on, but when it is run now, it returns only zero, since the GetPos() for the pmt hit collection class returns zero. This seems to have causes in the class definitions and so on - LXePMTHit.cc doesn't mention fPos private variable in the over-ride function for = operator and so on. Instead of trying to fix that, I just re-entered the position data in root, as

for (int i=0;i<8;i++)
    {
    y = -22.75;
    for (int j=0;j<8;j++)
      {
      posx[i][j]= x ;
      posy[i][j]= y ;

      //printf("pos2D[%d,%d]=%f,%f\n",i,j,posx[i][j] , posy[i][j] ); 
      y+=6.5;
      }
    x+=6.5;
    }

blogger limitations

Blogger is still raw around the edges! Surprising, even after so many years.

• A post I did with email - the one just before this one - became mangled somehow, and made any page containing that post to break the browser so that links were disabled. Deleted it and reposted it from blogger compose window in order to fix it. Not sure what exactly was the issue, perhaps links are the issue in emailed posts.
• Any < or > anywhere in the post has to be manually escaped, written as & lt ; and & gt ; respectively (without the spaces, of course). Otherwise blogger creates all sorts of problems, misrecognising them as html tags.

three interesting fulldome content-creation links

Rendering of 3D fractals, Mandelbulber - via LR.

Rendering of molecules and other datasets, Chimera - via fulldome yahoo group.

Rendering of circular symmetric shapes with polar co-ordinates in AfterEffects - via Newton's Cannon blog.

opening multiple files in ROOT

If the filenames (or pathnames in this case) are sequential, this method in the ROOT forums would work.

while (i1<50) {
   
 sprintf(filename1,"pathnameTemplete_%d.txt",i1);
    f1 = new TFile(filename1,"READONLY");

and so on.

Otherwise this glob based solution might be useful.

ROOT canvas save problem

At first I could not save the ROOT plots, error mesg
Error in <tunixsystem::dynamicpathname>: Postscript[.so | .dll | .dylib | .sl | .dl | .a] does not exist in <tunixsystem::dynamicpathname> ...

Then found this discussion about this very same issue.

sudo apt-get install libroot-graf2d-postscript5.34

Solved.

Now I can save as png or pdf / ps etc. But not gif. Error,
Error in <tasimage::writeimage>: error writing file

But pdf and png are fine for now :)

data analysis with ROOT

Got similar results to my previous post with Octave, using ROOT.

Had some hiccups. Initial testing of ROOT C macro failed due to that particular sample data file being empty! The file input was based on this basic.C example. After getting the samples into the ntuple,
while (1) {
      in >> runID >> sID >> energy >> pixelID;
      if (!in.good()) break;
      ntuple->Fill(runID,sID,energy,pixelID);
      nlines++;
   }
I could do a simple linear plot just to check,
ntuple->Draw("pixelID","energy");

Then I created the same sort of 2D matrix as in octave - perhaps this is not necessary, but this is the way I did it :)

 for (int i=0;i<8;i++)
    for (int j=0;j<8;j++)
     {
     energy2D[i][j]=0;    // initializing to zero, since 
                          // some i,j are not mentioned in the next loop
     }

for (int i=0;i<nlines;i++)     {
     ntuple->GetEntry(i);
    //pmtNumber pmtx is an 8x8 array,
    // assumed numbered rowwise.
    //for i=0:7, j=0:7,
    //i*8+j=pmtx. So
    jj= int(pmtx) % 8; //here we need to cast the float pmtx as an int
    ii= (pmtx -jj) / 8;
    energy2D[ii][jj]=enx   
     }

for (int i=0;i<8;i++)
 for (int j=0;j<8;j++)      {
     h2->Fill(i,j,energy2D[i][j]);
     }
h2->Draw("lego2");