Tuesday, February 06, 2018

creating a dashboard

One way of creating a dashboard for an organisation spread over different campuses, who have disparate IT setups - creating a google site with google docs or charts embedded from each campus - each campus can modify their charts separately, and the modified charts are reflected in the main site.

An example is at

Sunday, December 17, 2017

AutoSSL feature of WHM

Found that WHM has an AutoSSL feature, which will automatically configure SSL for selected (or all) users. The AutoSSL feature can be used with LetsEncrypt.org also, by running the shell script  /scripts/install_lets_encrypt_autossl_provider
as root, as mentioned in the documentation.

So, no need of going through the painful process of WHM SSL certificate installation.

And if we want to redirect all users using http to https, we can use the htaccess file, as given here

Sunday, November 26, 2017

compiling siril

Following the dependencies listed at
I did the following to install dependencies on Linux Mint 18.1 -

sudo apt-get install gtk3*
sudo apt-get install cfitsio*
sudo apt-get install fftw3*
sudo apt-get install gsl*
sudo apt-get install libconfig++*
sudo apt-get install libraw*
sudo apt-get install libffms2*
sudo apt-get install libtiff*
sudo apt-get install libjpeg*
sudo apt-get install libpng*

Then running autogen.sh complained -
./autogen.sh: intltoolize: not found

and for that I did
sudo apt-get install intltool
sudo apt-get install libgsl*

This was enough to build it using
 sudo make install

Thought about doing a full-fledged fork and pull request as in

For that, I needed to set the ssh key, as in this stackoverflow post,

cat ~/.ssh/id_rsa.pub
and copied the key to github -> settings -> ssh key

git clone git@github.com:my-account-name/Siril-0.9.git
cd Si*
git remote add upstream git@github.com:lock042/Siril-0.9.git
git checkout master
git pull upstream master && git push origin master
git checkout -b feature/stackallmedian

It turned out that this version had a bug where stackall would cause a segmentation fault. This was fixed in version v0.9.8-beta3 which also had a stackall median command, which did not do normalization by default, so my problem was solved and I did not have code it myself to do the git pull requests and so on.

Sunday, November 05, 2017

more on SER, FITS and Octave

Adding to my earlier post - Siril had a stackall command which stacks all the SER files in a folder to sum-stacked FITS files. That would have taken away the drudgery of having to open 830 SER files manually one by one and converting them though the creation of directories could be automated by

mkdir $(seq --format 's%03.0f' 1 830)

https://unix.stackexchange.com/questions/48750/creating-numerous-directories-using-mkdir )

which would create
etc upto

Unfortunately, Siril's stackall does sum stacking, and (at least according to my understanding, in version 0.9.7) was integer overflowing or showing some such behaviour - though the developers mentioned that it is supposed to convert to 64 bit, and then normalized to 16 bit. Maybe the normalizing was what was causing the issue. My data set had numbers like 44,000 per pixel and 102 frames, resulting in sums like 4,400,000 - even signed 32 bit int would only overflow at 2,147,483,647. Anyway, using the stackall sum stacking, I was getting plots like this

instead of plots like this.
Possible solution may be to implement a median stacking function for siril, like stackallmedian.

Edit: The siril team has now implemented median stacking using the command stackall median - and current branch has no normalization applied, so it works well for me :)

Friday, November 03, 2017

compiling QHYCCD camera SDK sample application with OpenCV

Steps needed to cleanly compile the LiveFrameSample code from QHY camera SDK, once OpenCV is installed -

Tested the opencv installation, compiling and running webcam capture test code from http://henryhermawan.blogspot.in/2007/09/capturing-frames-from-usb-camera-using.html

Followed the INSTALL file and compiled + installed the QHYCCD SDK.

sudo make install

Tried the sample without modifications -
cd sample
mkdir build
cd build
cmake ..

LiveFrameSampletest, SingleFrameSampletest and ControlCFWtest compiled without errors, but for CaptureDarkFrametest errors were reported, like
CaptureDarkFrame.cpp:76:83: error: invalid conversion from ‘int*’ to ‘uint32_t* {aka unsigned int*}’ [-fpermissive]
         ret = GetQHYCCDChipInfo(camhandle,&chipw,&chiph,&w,&h,&pixelw,&pixelh,&bpp);

and so on.

Anyway, I was only interested in the frame capture tests, so I created LiveFrameSampleOCV.cpp from LiveFrameSample.cpp and made the following changes -

  1. Uncommented the #define for opencv support and made that line
    #define OPENCV_SUPPORT 1
  2. Changed the set resolution line to
    ret = SetQHYCCDResolution(camhandle,320,240, 320, 240);
    (first two arguments after camhandle are dimensions, next two are position)
  3. Defined an int key before the display loop, and changed the display loop to an infinite loop -
    //ret = QHYCCD_ERROR;
    //while(ret != QHYCCD_SUCCESS)
  4. Added a break on escape key,
    if (key == 27)
  5. Modified the CMakeLists.txt to uncomment
    find_package(OpenCV REQUIRED) and

    and also changed the link libraries line to
    target_link_libraries(${FILENAME1} -lqhy ${LIBUSB_1_LIBRARIES} ${OpenCV_LIBS}
    where the OpenCV_LIBS have been added. 
  6. Ran cmake .. from the build directory again, and then
    make LiveFrameSampleOCVtest
So now ./LiveFrameSampleOCVtest shows a live display of the camera at 320x240 resolution, around 20-24 fps. 

While running cmake, it shows
 -- Found libusb-1.0:
--  - Includes: /usr/include/libusb-1.0
--  - Libraries: /usr/lib/x86_64-linux-gnu/libusb-1.0.so

But could not build LiveFrameSample without using cmake - the gcc command needs some more work - this does not work -

gcc -Wall LiveFrameSampleOCV.cpp -o LiveFrameSampleOCV -l/usr/lib/x86_64-linux-gnu/ `pkg-config --cflags --libs opencv usb-1.0`

installing the latest OpenCV with Python bindings on Linux Mint 18.1

Followed the Ubuntu 16.04 tutorial by Adrian Rosebrock with some changes. Since my installation had python using anaconda, I used conda install instead of pip install. And since my first attempt at creating a virtual environment failed, I went ahead with the installation without a virtual environment. Most of the requirements were already installed. My cmake string also had a different path to the python executable due to anaconda.

sudo apt-get update 
sudo apt-get upgrade
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg8-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev 
sudo apt-get install libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev python3.5-dev

OpenCV stable version is now 3.3.1, so
wget -O opencv.zip https://github.com/Itseez/opencv/archive/3.3.1.zip
unzip opencv.zip
wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/3.3.1.zip
unzip opencv_contrib.zip
conda install numpy
cd opencv-3.1.0/
mkdir build
cd build
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-3.3.1/modules \
    -D PYTHON_EXECUTABLE=/home/mac/anaconda2/bin/python \
cmake rapidly went through its checks, then I had to do the build,
which took around half an hour judging by the timestamps - I was doing other things.... and then finally 

sudo make install
sudo ldconfig
Verified by doing
ls -l /usr/local/lib/python2.7/site-packages/
which showed up cv2.so
and finally linked this cv2.so to the site-packages folder, in my case
cd ~/anaconda2/lib/python2.7/site-packages
ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so
Tested by opening a python shell, importing cv2 and checking its version,

 >>> import cv2
>>> cv2.__version__


Monday, October 30, 2017

SER, FITS and Octave

While capturing a series of images in an autocorrelation experiment using a QHY5L-II and Sharpcap, instead of capturing as a series of PNG files, captured as SER files by mistake. From the experience of trying to use that data, found the following.

  1. SER capture is faster than PNG sequence capture - PNG was capturing at 20 fps or so, but dropping frames after 80 frames or so, and was taking the last 20 frames at around 5 fps. SER was not dropping frames at all. All this on the Menlo systems Lenovo laptop. 
  2. Octave and Matlab don't open SER files without conversion, the definitive site for SER files seems to be http://www.grischa-hahn.homepage.t-online.de/astro/ser/
  3. After some back and forth on this thread, found that Siril can export SER to a sequence of FITS images without losing the 16-bit resolution. 
  4. Imagemagick can be used to convert the FITS images to 16-bit TIFF using
    mogrify -format tif *.fits
  5. Alternately, the fits octave package can be used to read the data into Octave from the FITS files. Although Octave uses Imagemagick to implement imread, imread doesn't seem to work with the FITS files from Siril, complaining
    error: Magick++ exception: Magick: Unexpected end-of-file (/media/mac/SAM2TB/Autocorr/2017-10-16/Capture/14_10_49/Capture_00002.fit) reported by coders/fits.c:370 (ReadFITSImage)
  6. In order to install the fits package, had to install its dependency, with
    sudo apt-get install libcfitsio*
    and then run octave as root,
    sudo octave
    and within octave, run
    pkg install -forge fits
  7. Reading using the fits package and imread give different results.
    imread returns uint16 rows x columns
    read_fits_image returns double columns x rows, upside down.
    So, for equivalence with imread, we need to use flipud.

    gives the same numbers as
  8. identify Capture_0035.png
    can be used to check whether the png is 8-bit or 16-bit. 
  9. Installing ISIS and SER Player turned out to be dead-ends, since they did not convert SER to PNG without converting to 8-bit.
Edit - more on siril's stackall here

Saturday, October 28, 2017

removing malware/adware from Intex phone - trois

Even after my earlier posts about removing malware from Intex phone, one issue persisted. Occasionally, Rs. 1.50 would be deducted as SMS charges. At first I thought it was some sort of location finding behaviour by some app - as I noted, this deduction was noticed after I installed Uber and PayTM apps. But even after uninstalling those apps, the issue remained - deduction of Rs. 1.50 once in a week or so.

Finally I contacted my carrier Airtel via email - which seems to be the only way to reach their customer service department. They advised that I could Dial *121*51# (Toll Free) or 12181 (Toll Free) to check last 5 deductions which have happened in the last 72 hours. Using this information, I found that the SMS charges were to a number 9582943043. Googling, found that this is an "activation feature" on Intex phones!


Now I seem to have removed the offending Intex services - using Kingroot and root uninstaller - searched for sms in system apps, found SmsRegister - com.android.smsregister - uninstalled it. This seems to have solved the issue - no sms sent for the last one month.