tool for converting 360 videos to fulldome - OCVWarp and ffmpeg's v360 filter

Paul Bourke had recently announced a real-time video playback tool which can convert and play equirectangular 360 videos as fisheye videos on domes like planetariums. 

OCVWarp can currently convert equirectangular 360 videos to fisheye videos offline - fisheye videos with 180 degree field of view would be suitable for projecting in a planetarium - fulldome videos. While the field of view is fixed at 180 degrees, the direction of viewing can be changed interactively, or by setting parameters in the ini file. This tool is useful for those who have lower powered machines which cannot handle hi-res video processing in real-time, and also for Windows and Linux users.

Here are links to OCVWarp's
Source code
Linux AppImage
Windows commandline-only executable with some restrictions.

Usage notes and bare basic compile instructions are in the readme.

Some examples, using input files found using google - these are transformations from Equirectangular to 180 degree fisheye, Type=1 in the ini file.

Input -

Output with constant parameters of anglex=0 and angley=-90 -

Output with anglex being changed by pressing 'h' once every second or so -

Output with anglex changed every frame, by keeping 'h' pressed -

Recalculation of the map files is quite slow - on my machine, 1 frame per second for 1024x1024 output. But in non-interactive mode, uncompressed 1024x1024 video was being output at around 22 fps. Encoding to some other codec would probably have slowed it down to 10 fps or so, similar to what ffmpeg was doing. 

ffmpeg's newly introduced v360 filter is supposed to do similar transformations, but it's not interactive.

Note: Paul adds in the comments below that there is a way to change the transformations over time with the v360 filter.

For testing v360 filter with ffmpeg, I had to download the static build for 24th March 2020, since even the latest release did not have this functionality.   

ffmpeg's output on the left, OCVWarp on the right. The default choices for interpolation seem to favour OCVWarp for better clarity. (Click on image for a bigger view.)

ffmpeg's output -

The ffmpeg commandline used was
./ffmpeg -i stock.webm -vf v360=e:fisheye:v_fov=180:h_fov=180:yaw=-90:w=1024:h=1024 outputfovsc-90.mp4
and a nice ffmpeg tutorial is available here.

And here is a link to all my posts about OCVWarp.

Edit on 2nd April - New release, with option to change output codec.

tool for converting 360 videos to fulldome - making of OCVWarp

I'd previously posted about OCVWarpNorth. Now, OCVWarp is finally ready with Equirectangular to 180 degree Fisheye conversion of arbitrary fields of view.

The delay was due to bugs, which I finally squashed by getting a better understanding of the transformation equations using Octave.

I'd intuitively assumed that applying a rotation tranformation to Px, Py, Pz in http://paulbourke.net/dome/dualfish2sphere/diagram.pdf should do the trick for getting any field of view. But I was done in by a bug introduced fairly stupidly, when I had specified the same variables on both sides of the rotation, like
Py = cos(angleyrad) * Py - sin(angleyrad) * Pz;
Pz = sin(angleyrad) * Py + cos(angleyrad) * Pz; 
// here, Pz gets calculated with the new Py 
//instead of with the old Py - this is the bug
instead of
PyR = cos(angleyrad) * Py - sin(angleyrad) * Pz;
PzR = sin(angleyrad) * Py + cos(angleyrad) * Pz;

I'll prepare a separate post comparing OCVWarp and ffmpeg's v360 filter.

One of the interesting views obtained during the learning process with Octave - from equidist.m, after a rotation -

This image gives some idea of the interpolation going on, especially for a low-res image - the black areas are non-interpolated, etc.

While reading up to do debugging, I found the following quite useful -

• code from pano2fisheye and fisheye2pano
• references and explanations at http://paulbourke.net/dome/fisheye/ - especially the hemispherical fisheye vs angular fisheye and so on.
• the simplest orthographic projection from Wikipedia
• Map projections at Wolfram's Mathworld
• Transformation matrices at Wikipedia
• From the 3D projection page, the Parallel projection page

and finally, of course, just doing tests in Octave. 

online mastering

An option if just a single track needs mastering and you don't have Reaper or any other tools - online free mastering via https://www.bandlab.com/mastering

creating subtitles with Aegisub

An updated version of this old post, using the powerful cross-platform tool Aegisub instead of DivXLand Media Subtitler.

1. Create subtitles in a plain text editor, one line per subtitle, save as a .txt file.
   
2. Load the video (Video -> Open video ...) and the txt file (File -> Open subtitles...) into Aegisub using the respective menu items.
   
3. Open the waveform for the timing view (Audio -> Open Audio from video). I generally find the waveform view (Audio -> Waveform display) to be intuitively understandable.
   
4. Inside the waveform view, left click at the beginning of the point where you want your first subtitle. A red line appears for the begin time. Move the mouse over the blue line indicating the end of the subtitle, and when the left-right arrow appears, click and drag to move the end time.
   
   
   
   
   
5. Once the timing is right, commit by pressing the 'g' key.
   
6. Keyboard shortcut keys make the process much faster.
   s to play the selected portion (between begin and end)
   v to move the end time more to the right
   c to move the begin time more to the left
   g to commit (writes the subtitle begin and end time, and moves to the next line)
   
7. If useful, follow these tips for speeding up your work.
   
8. At the end, File -> Export subtitles. Can usually leave the settings as the default, and can export as filename.srt for a common subtitle format which is accepted by youtube etc.
   
9. Youtube detailed instructions for adding subtitles is under "Upload a file" section in this support page.
   
10. For local playback, VLC, for example can open the subtitle file using the menu Subtitle -> Add subtitle file.

 

ways to export layers from Gimp

From

https://askubuntu.com/questions/485917/gimp-export-every-layer-as-a-separate-png-image

 

If PNG is an acceptable output format, one option is to export it as Open Raster (.ora), an open specification for layered-image files.

1. Export Image as Open Raster (.ora)
   File -> Export As ...
   myfile.ora
2. Open myfile.ora as an archive, with a program like file-roller or 7zip.
   On Ubuntu:
   

   $ file-roller myfile.ora
   

   (alternatively)
   

   $ unzip myfile.ora
   

   All your layers will be png images under /data, Extract them and use at will.
   
   Apparently Gimp 2.10 onwards, ORA support is dropped.

find large files in google drive

This article https://www.theverge.com/2019/3/19/18271015/google-drive-photos-storage-space-how-to-get-more has a link to google drive's (undocumented) quota feature, which lists files in order of size,

https://drive.google.com/drive/u/0/quota

Along with the search terms in gmail for attachment size, like
larger:2MB smaller:10MB
which can find emails of a particular size range, this would be useful for people to manage their google quotas.

PHP odbc support issue

Shared by PB - a 3-day exchange of emails with Nettigritty support for the following:

(our site) which runs on PHP 5.6.40 has x86 architecture with ODBC Library Win32
(our beta site) which runs on PHP 7.4.3 has x64 architecture with ODBC Library as Win32 (should have been Win64)

We request you to install either of the following options:

 

1. Win64 version of the ODBC library for PHP 7.4.3 (64 bit)

        or

2. Win32 version of the ODBC library for PHP 7.4.3 (32-bit)

We get the error odbc_connect(): SQL error: [Microsoft][ODBC Driver Manager] The specified DSN contains an architecture mismatch between the Driver and Application, SQL state IM014 in SQLConnect

PB also shared that we are using a connect-string like
$db_connection_variable = odbc_connect("Driver={Microsoft Access Driver (*.mdb)};Dbq=$dbq", '', '', SQL_CUR_USE_ODBC);

and that Plesk logs for our domain show the error

odbc_connect(): SQL error: [Microsoft][ODBC Driver Manager] The specified DSN contains an architecture mismatch between the Driver and Application, SQL state IM014 in SQLConnect

Nettigritty support replied, "Please try with a DSN-less connection. You can refer the link below for an example 
https://webcheatsheet.com/php/odbc_connection.php "

PB - We tried using a DSN-less connection as referred to you in the example link (https://webcheatsheet.com/php/odbc_connection.php)

We are getting the following error in the logs of (our beta site)

Code:

//create an instance of the  ADO connection object
$conn = new COM ("ADODB.Connection")
  or die("Cannot start ADO");

Error:

PHP Fatal error: Uncaught Error: Class 'COM' not found in C:\inetpub\vhosts\path\open_connection.inc:13
Stack trace: etc

You would need COM  to be enabled in the PHP configuration

When we use the DSN connection we get the a similar error as described here

https://stackoverflow.com/questions/26290532/data-source-name-not-found-and-no-default-driver-specified

Code:

$db_connection = odbc_connect("Driver={Microsoft Access Driver (*.mdb)};DBQ=$dbq", '', '', SQL_CUR_USE_ODBC);

Error:

PHP Warning: odbc_connect(): SQL error: [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified, SQL state IM002 in SQLConnect in C:\inetpub\vhosts\radiosai.org\httpdocs\tfd7\open_connection.inc on line 16

PHP Warning: odbc_connect(): SQL error: [Microsoft][ODBC Driver Manager] The specified DSN contains an architecture mismatch between the Driver and Application, SQL state IM014 in SQLConnect in C:\inetpub\vhosts\radiosai.org\httpdocs\tfd7\open_connection.inc on line 16

Kindly configure the system as per the suggestion provided here:

https://stackoverflow.com/questions/26290532/data-source-name-not-found-and-no-default-driver-specified

 Nettigritty - ... please find various types of connection strings for accessing Access details at https://www.connectionstrings.com/access/ and https://www.php.net/manual/en/function.odbc-connect.php

 Please note that the server already has ODBC drivers already installed and a screenshot for the same was already shared you to confirm the same. Please note.

 We recommend even switching your MS Access database to MSSQL database for better security and compatibility with newer scripts. ...

PB replied that we have already tried all the links etc. Finally, Nettigritty replied, 

we have enabled COM class for you for PHP and you could see the error for your application directly from https://beta.radiosai.org/tfd7/login.php URL itself now.

 

On checking further, we find the DSN created through Plesk is being created as 32bit only.

 

Either it is required to use the application in 32bit or we can delete and try adding the DSN directly from server under 64 bit to see if that helps. But we recommend doing the application in 32bit as creating the DSN through plesk only will be active and migrate when there is any OS upgrade at a later point of time.

Regarding installing 64-bit ODBC driver, they said, 
This is a pending feature requested to Plesk on Windows.

https://plesk.uservoice.com/forums/184549-feature-suggestions/suggestions/39056515-add-ability-to-select-64-bit-drivers-in-odbc-data

 

Presently, please share details to manually recreate your DSN on 64 bit ODBC on the server.

They manually created the system DSN, and then the connection worked for PB.

360 videos - online and offline

A short review of what works on my phone, LG Q6. Five years ago, I had done a similar review here and an Oculus Rift mini review here.

Downloaded hi res videos don't work. Probably the hardware does not support. This includes the 4K files from youtube, 3840x2048 H.264 High Profile.

Youtube app - 1440S works, online. Can use with google cardboard etc.

VR player - does not render the downloaded 360 video as a 360 video. Simply splits into two SBS screens.

From the list at

https://www.quora.com/Is-there-a-360-video-player-for-android-which-can-play-offline-videos-using-Gyrosensor-like-Google-spotlight

VR gesture player - does play the downloaded 360 files as 360 video. The gesture interface is a bit tricky to navigate, and if you don't want to use the gesture interface, the interface is tiny.

VR Player FREE - this one has a relatively simple interface, has the option to change the field of view and projections. Works for the videos which the phone supports. So, in my case, downloaded 1440S files don't work, but downloaded 720S files work.

Edit - on the Mi5, 1440S as well as 3840x2048 H.264 High Profile work fine. Even downloaded 4096x2048 video works fine. On VR Player FREE, the default settings caused the back of the screen to be blank. That is, if I turn around, I see a blank wall behind me in VR. Not sure if any setting change would give better results. Projection choices like plane, dome, fulldome etc were a bit confusing.

KMPlayer VR - the view was good on the Mi5. But the video itself did not play smoothly, was going at only a couple of frames per second or freezing.

VR Theatre for cardboard - gave excellent results for 360 video, with the settings Screen type - panoramic 360 and 3D encoding if any - Monoscopic for the 360 video I had downloaded. A good feature is that it asks for these settings for each file before playing.

Viewing the 360 video on google cardboard worked well.

On a low-cost VR-box, the view was a bit out of focus for me, and also slightly more restricted than in the cardboard viewer.

procedure for redeeming points for flights

As of now, March 2020, this is the procedure for redeeming points for flight bookings with HDFC Regalia and Regalia First credit cards.

Go to

https://offers.smartbuy.hdfcbank.com/

Click on the Privileges -> Regalia menu item.

Log in, do the flight booking, and at the end of the booking, just before the payment, option to pay with points appears. Rs. 99 is charged for generating a voucher to pay with points. Regalia First card holders can only pay 70% of the total using points. Move the slider all the way to the right (or upto 70% - in my case, all the way to the right was 70%).

Proceeding further, an email is sent with voucher code. We have to manually enter the voucher code and then make the payment with the HDFC Regalia card at the final step.

Thursday, March 05, 2020

caution when checking domainkey DKIM DNS records

I made a mistake when checking if a particular domain has DKIM records set - I just queried for TXT records on the domain, like
dig domainname.com TXT

That would return SPF records, but not domainkey DKIM records, since DKIM records can have arbitrary names, so looking at the DNS control panel is the better solution. And then can double-check when the DKIM record name is known, like
dig default._domainkey.domainname.com TXT
or
dig providername._domainkey.domainname.com TXT
and so on.

Monday, March 02, 2020

tool for converting 360 videos to fulldome for planetarium projection and Azure

Earlier, I posted about converting 360 videos to fulldome for planetarium projection using Blender. But this was excruciatingly slow. Then I worked on a tool to do a generalized warping (or remapping from one co-ordinate system to another) using OpenCV and CPU-based calculations - OCVWarp. This is still a work in progress, but some parts of it work - mapping to a 180 degree or 360 degree fisheye centred on the "North pole" of the equirectangular image is working as of now. Also with a simplified commandline-only tool to run on headless cloud servers. Meanwhile, found that ffmpeg also now has an option to do this directly, without the remap filter. Must explore the speeds etc.

On my i7 Macbook with only 4 GB RAM, output sizes greater than 2048x2048 (or maybe 3072x3072) was causing the process to run out of physical memory and start to write to swap disk. 2K output was running at around 4 fps. Tried running on an Azure VM with 16 GB RAM and on a borrowed Dell laptop with 8 GB RAM. Both of these could comfortably output 4096x4096 video at around 2 or 3 fps.

A quick note about the Azure VM. A Windows 10 VM took around 15 minutes to provision and start running. An Ubuntu 18.04 server took around 2 minutes or less.

Edit - OCVWarp has been updated. Here is a link to all my posts about OCVWarp.

Sunday, March 01, 2020

point to note for colour support with OpenCV and QHYCCD SDK

For adding colour camera support for ViewportSaver, learnt from the example code here that I had to add the crucial function which does de-bayering for QHY cameras.
SetQHYCCDDebayerOnOff(camhandle,true);

Without that, the output image was distorted, interleaved, greyscale. Further, we have to remember that OpenCV uses BGR as the colour order. So does QHY, so no issues with that. Because of this,
split(mraw, BGR);
//by default opencv puts channels in BGR order
BGR[1] = Mat::zeros(mraw.rows, mraw.cols, CV_8UC1); // green channel is set to 0
BGR[0] = Mat::zeros(mraw.rows, mraw.cols, CV_8UC1); // blue channel is set to 0
// and so on.

Subscribe to: Posts (Atom)