Tuesday, July 16, 2019

using DMCTK with CMake on Linux

This stackoverflow post mentions an old version of CMake and making on Windows. More googling led to FindDCMTK on cmake.org  so that I could just edit my CMakelists.txt to read

...snip...
find_package(USB-1 REQUIRED)
find_package(OpenCV REQUIRED)
find_package(DCMTK REQUIRED)

include_directories($(USB_1_INCLUDE_DIR))
include_directories($(OpenCV_INCLUDE_DIR))
include_directories($(DCMTK_INCLUDE_DIRS))

... snip ...
target_link_libraries(${FILENAME1} ${LIBUSB_1_LIBRARIES} ${OpenCV_LIBS} ${DCMTK_LIBRARIES} )
 ... snip ...

That worked nicely, without having to manually find the libraries and so on.

Tuesday, July 02, 2019

mounting a windows smb share and automating file moves

mounted //PATH/FOLDER on a linux machine onto a folder /path/to/mountdir using

sudo mount -t cifs //192.168.ip.address/FOLDER path/to/mountdir -o uid=usernameofowner,username=username,password=xxxxx

A cron job was added to automatically move files every 5 minutes, like
*/5 * * * * mv /path/to/mountdir/LIVE_*  /path/to/another/dir/

But that caused problems, generating hundreds of emails to usernameofowner, saying mv: cannot stat ‘/path/to/mountdir/LIVE_*’: No such file or directory whenever there was no file there to move. So, PB modified the cron by adding
> /dev/null 2>&1 
which seems to have solved the issue, working well.

Wednesday, June 12, 2019

income tax filing quirks for assessment year 2019-20

This year, I thought of filing ITR1, since I only had short-term capital losses and no gains. But on downloading the pre-filled XML and trying to pre-fill the java utility, it gave an error that the xml is not valid. Then tried ITR2 java utility, with ITR2 prefilled XML. This time it mentioned that the assessment year is wrong. Manually edited 2018 to 2019 in the XML, and it pre-filled successfully.

ITR2 needed Basic Salary and DA. So, I computed it from my salary slips, as a percentage of gross salary.

Unfortunately, the CG (Capital Gains) section was not working properly in the java utility (AY2019-20 version 1.2). The Mutual Fund section's form was not being displayed, only the instructions were being displayed. So, I did not put in my losses to carry forward to the next year.

The tax computation is in the tab Part B TTI. When you click on Save, it does validation. This time, there was also a Submit button, which allows you to directly submit the return from the java utility itself. I did that, and the ITR-V was sent to my email. I chose to e-verify via HDFC bank's netbanking like in this earlier post. The first time I tried, I was logged on to the incometax portal when I started the process, so it did not work. I needed to log out of the incometax portal, and then click on the Income tax e-filing link under Requests in HDFC bank's netbanking. Then it worked. 

Wednesday, May 29, 2019

ffmpeg commandline for maximum compatibility

An old Raspberry Pi was not playing some videos. Looked around for a transcoding option with ffmpeg, baseline x264 seemed fine, and this example worked for me -
ffmpeg -i input.avi -c:v libx264 -preset slow -crf 22 -c:a copy -profile:v baseline -level 3.0 output.mp4
by looking at

Another post has the 2-pass encoding example, where the bitrate is also specified. 

Tuesday, May 21, 2019

videos for waiting area - redux

The original idea of having a marquee which would update as in my previous post turned out to be not very workable. Even if the wifi was not flaky, updating the times etc on a computer in the office would be "too much work" for us in the middle of shows. And the wifi signal was being completely lost due to electrical noise from our solar system exhibit's motor and my room's AC compressor motor. So, the idea was adapted to showing a notice in 3 languages, that the average wait time is 30 minutes, every five minutes or so.

Notice made in Google docs and screenshotted, added an "invert" filter scrolling up-down for visibility.

The next change was to use the "smart TV" media playback for the two flat screens instead of buying Raspberry Pis for them. Since anyway the content was going to be static, did not make sense to buy. Idea suggested by S.

The "smart" TVs would only play a single video file, would need intervention after a single video file was played, would not play playlists. So, strung together all the files into a single file, repeated it for 3 hours so that no intervention would be needed. I had used avidemux, vdub, ffmpeg for doing this, but S more comfortable with VSDC on Windows.

Ffmpeg commandline for making files of known size - first calculate required bitrate by size / duration, using kbits and seconds, then something like
for half an hour 500 MB, = 500 * 1000 * 8 kbits
bitrate = file size / duration
=4000000/1800 = 2222 kbps. Let's take as 2200 kbps.
so, 2200k

ffmpeg -y -i input -c:v libx264 -preset medium -b:v 2200k -pass 1 -c:a libmp3lame -b:a 128k -f mp4 /dev/null && \ 
ffmpeg -i input -c:v libx264 -preset medium -b:v 2200k -pass 2 -c:a libmp3lame -b:a 128k output.mp4

In the ffmpeg example obtained by googling, it was AAC audio. But on my machine,
 Unknown encoder 'libfdk_aac'
So, used libmp3lame instead.

Playing the single file on the Pis had some issues. The older Pi would not play, since it did not support the codec -  x264 High Profile, I think. The other Pi had vignetting issues, the 16x9 videos were appearing too small on the 4x3 TVs. So, the Pis play playlists, and the "smart" TVs play single video files. The older Pi does not play VP8. Files downloaded from Youtube via desktop (MPEG4 codec) were fine, downloaded via mobile by S were VP8.

Also learned that the Raspberry Pi overscan controls do not work with VLC in fullscreen mode (on RPi3).

Also learned that the Raspberry Pi (2 and 3) video out pinout of the TRRS connector did not have Sleeve as ground! TRRS = L, R, GND, Video! Had to modify the TRRS to RCA cable we bought, inverting GND and VID for one of the RCAs.

For getting composite video out, 0 is NTSC and 2 is PAL,
https://www.raspberrypi.org/documentation/configuration/config-txt/video.md

if only sdtv_out=2 does not work,

https://bhavyanshu.me/tutorials/force-raspberry-pi-output-to-composite-video-instead-of-hdmi/03/03/2014/

And with the remote for shutting down the Pis, they can be operated completely independent of wifi availability.

Sunday, May 12, 2019

Veracrypt fix

A Veracrypt volume on a removable (NTFS) hard disk was not mounting. Veracrypt was complaining of bad password or bad volume. I thought the password was correct. Finally, did
sudo fdisk -l
to find the device name, which was /dev/sdc1 for the removable drive in this case, and then
sudo ntfsfix /dev/sdc1
It finished in less than a second, noting that it had successfully completed. Then, tried mounting with Veracrypt, success!

After mounting, the encrypted volume was visible in Nemo with an icon to unmount it. But unmounting it with Nemo does not really unmount it, probably because Veracrypt uses su privileges to mount - you have to unmount (or Dismount) it via the Veracrypt interface. Only then would the removable drive become free of processes and unmountable via Nemo.



Thursday, May 02, 2019

Octave memory issues and workaround

I had an Octave script which was reading data from files into a variable, doing some processing, then reading some different data into the same variable, and so on. For many hundreds of MB of data, using the load function. But Octave was running out of memory, and finally I had to
killall -9 octave-gui

The workaround was to process part of the data, save the intermediate data to file, exit Octave, reload Octave, resume processing, and so on. Painful, but it worked. 

Wednesday, April 24, 2019

chapterwise references with LaTEX and biblatex

I needed chapterwise references for my thesis. The suggestions at
https://tex.stackexchange.com/questions/345163/references-at-the-end-of-each-chapter
did not work with the "Cambridge" template on Overleaf.

or use TU Delft template which has per chapter references.


used that, and found:

The chapterbib package and biblatex are incompatible

Then got this - 

That seemed to work - \begin refsection etc 

% Chapter 1
\begin{refsection}
\chapter{Introduction} % Main chapter title

---snip----

% and at the end, where we want the references

\renewcommand{\bibname}{References}
% 20190501 - HN
% added this renewcommand to change  the title Bibliography to References
\printbibliography
\end{refsection}

Tuesday, April 16, 2019

mailstore for achiving emails

Here is part of an email to a colleague regarding Google Apps email reaching storage limits:

Regarding email - currently, our domain does not allow expansion of the existing amount of space. So, the only way is to delete large attachments. In many cases, the attachments are required and cannot be deleted. This can be handled in many ways.

1. Forward them themewise to specially created gmail ids and then delete
2. archive them locally by storing on a local hard disk and then delete
3. automate the local archiving using some tool like
https://www.google.com/search?q=imap+archive+tool+open+source

I guess you would want to work on Windows? I have not tried any of these tools, but this free Windows tool comes near the top in search results -
https://www.mailstore.com/en/products/mailstore-home/

And later, when he tried out mailstore, found that it had a specific option to get authorized for GMail - using that option, downloading emails was quite painless, though a long process for many GB of emails.

Thursday, April 11, 2019

voting in India

Today is polling day for us. Both State and National elections being held simultaneously here. Though the printed slips with our electoral roll serial number and polling booth location were given to us from our workplace, it is also available for all as an app,
https://play.google.com/store/apps/details?id=com.eci.citizen&hl=en_IN

Lots of information is available online as well as on the app, for example, the eligibility criteria and how to register,
https://eci.gov.in/faqs/

My experience was relatively seamless. Waited for around 20 minutes in line - around 20 people in line in front of me, including elderly ladies who had to be prompted on how to use the machine etc. All done in a harmonious manner. Very civilized. A demo video is at
https://eci.gov.in/video-gallery/evm-awareness/know-your-evm-r61/

Friday, April 05, 2019

Raspberry Pi shutdown using IR remote

Initially followed this instructable -

Had to make changes because of new version of Raspbian and lirc.

1. Added sensor as per this,
changing pinout as required by googling
2. Installation by
sudo apt-get update
sudo apt-get install lirc



3. Configuring kernel module was different from
The only thing we need to do is to uncomment out
dtoverlay=lirc-rpi
in /boot/config.txt

4. Configuration was slightly different from
(a) create /etc/lirc/hardware.conf and put in it
LOAD_MODULES=true
DRIVER="devinput"
DEVICE="/dev/lirc0"
(b) Placing the remote's configuration file was inside the directory /etc/lirc/lircd.conf.d
(c) starting the service was by /etc/init.d/lircd start

5. irexec configuration was different from
We needed to edit /etc/lirc/irexec.lircrc instead, since we want to shutdown and reboot, editing it to:
begin
    prog   = irexec
    button = KEY_1
    config = sudo shutdown -r now
end
begin
    prog   = irexec
    button = KEY_0
    config = sudo shutdown -h now
end


6. irrecord was a little tricky -
(Here also, instead of service lircd stop, we need to do /etc/init.d/lircd stop)

irrecord first wants you to press random keys. Then, you have to give the name of a key, press and hold it, then enter the next key's name when prompted, and so on for all the keys, and then only press ENTER to stop the process. And some keys are not recognized - maybe those should have been pressed during the first phase of random pressing.

7.  sudo /etc/init.d/lircd status reports error with duplicate remotes. We need to rename /etc/lircd/lircd.conf.d/devinput.lircd.conf to devinput.lircd.conf.dist
sudo mv devinput.lircd.conf devinput.lircd.conf.dist

8. sudo /etc/init.d/lircd status reports errors like
lircd-0.9.4c[364]: Error: Cannot glob /sys/class/rc/rc0/input[0-9]*/event[0-9]*
Then we need to edit /etc/lirc/lirc_options.conf, changing
device = devinput
to
device = default

9. sudo /etc/init.d/lircd status reports errors like
Error: could not get file information for /dev/lirc0
After a lot of searching, found on this page that this could be because the driver lirc-rpi has been replaced with a newer version called gpio-ir, so that the /boot/config.txt line has to be changed from
dtoverlay=lirc-rpi
to
dtoverlay=gpio-ir
Did that, and it worked.
This is on Raspberry Pi 2 running Raspbian 9 (Stretch) with kernel 4.19.23+, running lirc version 0.9.4c-9 while the other Pi, which worked with dtoverlay=lirc-rpi was running Raspbian 9 (Stretch) with kernel 4.14.98-v7+ and the same lirc version  0.9.4c-9.

testing AppImages on various distros - dd iso to USB

As mentioned in my previous post, I had created an AppImage for distribution a Linux binary. Testing on 3 distros one generation earlier is recommended using chroot. For example,
https://www.pcsuggest.com/setup-a-32-bit-chroot-with-ubuntu-live-cd/

But I found the procedure for creating chroots to be more cumbersome than just dumping the Live distro iso into a USB pen drive, booting another machine from USB, and checking the AppImage on the distro.

1. Since I was on an Ubuntu-based Mint Linux 18.1, it had a "Startup Disk Creator" GUI app for dumping Ubuntu-like Live CD/DVD to USB devices.

2. For Fedora and OpenSuse, I used the following commandline, taking care to note the correct device id like /dev/sdc1 or /dev/sdd1 or whatever using mount,
sudo umount /dev/sdd*
sudo dd if=Fedora-Workstation-Live-x86_64-24-1.2.iso of=/dev/sdd bs=8M status=progress oflag=direct

3. This commandline pattern worked fine for Fedora and OpenSuse, since these isos were meant to work as Live CD/DVDs. So, I did not need to use unetbootin or anything like that for these. Approximately 3 MB per second to my USB drives, which were not very fast. 7-8 minutes to write 1.5 GB. 

creating an AppImage to distribute Linux binaries

Some notes of my AppImage creation process.


  1. First tried creating manually, using
    https://github.com/AppImage/docs.appimage.org/blob/master/source/packaging-guide/manual.rst and the AppRunx86-64 from https://github.com/AppImage/AppImageKit/releases
  2. Found and copied the libs it uses by doing the following to copy all shared libraries -https://www.commandlinefu.com/commands/view/10238/copy-all-shared-libraries-for-a-binary-to-directory
    ldd file | grep "=> /" | awk '{print $3}' | xargs -I '{}' cp -v '{}' /destination
  3. Need to use the environment variable $OWD (Original Working Directory) to get files from the OWD, like our ini file. This code snippet was a good template, https://stackoverflow.com/questions/40306012/c-using-environment-variables-for-paths
        char* pPath;
        pPath = getenv("PATH");
        if (pPath)
          std::cout << "Path =" << pPath << std::endl;
        return 0;
  4. Many libs copied with the manual method had to be excluded using the excludelist at
    https://github.com/AppImage/pkg2appimage/blob/master/excludelist
  5. Still did not work on newer distros. In this thread, probono recommended using trusty on TravisCI for building, and linuxdeployqt instead of manual creation of AppDir. Did that, and it worked. Far fewer libs are bundled by linuxdeployqt. Lists of the two bundles are available at this issue thread.
  6. The download link from transfer.sh can be seen if the entire log is downloaded from TravisCI.
  7. The travis yml file gives the outline of the build and bundling process.

Thursday, April 04, 2019

USB drive reported as read only - solution

Found my USB drive reported as read-only on Linux Mint 18.1.

Corruption could cause this, so tried
sudo dosfsck -a /dev/sdc1

That did not help. Though the CLI is able to write to it, Nemo still complains it is read-only.

Apaprently the fix is at https://forums.linuxmint.com/viewtopic.php?t=176546

So everything needed to fix the issue is to execute:
sudo chmod 777 /media/USERNAME

And/or maybe a reboot?

When I faced this issue today, I rebooted the system, and then the USB drive was mounted read/write.

Wednesday, April 03, 2019

transparent proxy over ssh - sshuttle

Sometimes the network to which I am connected fails, while the LAN over fiber to Studio is active, and Studio's backup network is functional. Then, an elegant way to use the Studio network, without having to install any proxy server software, is using sshuttle.

sshuttle --dns -r username@ip.address.of.machine 0.0.0.0/0

will transparently proxy all dns requests as well as all traffic through the remote machine. Only python needs to be installed on the remote machine. 

Tuesday, March 19, 2019

Fedex (India) - expectation vs reality

Documenting my experience with Fedex. Plenty of lessons here for everyone, I think. My takeaway was, avoid Fedex (at least in India) whenever possible, if you are an individual, and not a corporate with a Fedex account. Perhaps they treat corporates better - who knows. But the level of miscommunication and lack of communication would indicate otherwise. Even our University had requested us a couple of years ago to avoid Fedex when importing.

1. I get a phone call on Saturday, 23 Feb 2019, that my shipment is due to arrive the next day, and I need to submit the required documents. Since I had already submitted my "one time only" Know Your Customer (KYC) documents last year, I thought this would be a breeze. I was wrong.

2. The form I had to fill in this year was different from last year's. (My printer was on the blink, so I had to trek all the way to another office, 15 minutes away, in order to print all the new documentation that had been requested.) The documents requested would probably change again, so it may not be worthwhile documenting them, but just for completeness:


·        KYC Form along with supporting documents – DULY SELF ATTESTED {signed & stamped}  (KYC is mandatory and is a 1 time requirement)

·         PAN copy,

·         Aadhar Card & Voter ID card

·         Address proof. (Telephone /Electricity Bill/Bank Statement/ Rental Agreement )

·         AD code ( Authorized dealer code from Bank)

·         DUMMY IEC LETTER

·         Gatt and DGFT declaration ( Attached )

·         Technical write up & INDIVIDUAL NET WEIGHT FOR ALL ITEM to prepare Advance checklist

3. At first, the Fedex rep on the phone told me that "since the item has come in cargo mode" it was mandatory that I have an IEC code - which is an Import-Export code for importers. She told me to find out from my organisation if I can import under their IEC, saying that individuals cannot import. Though I tried telling her that I had imported items in the past, she said it was impossible. Luckily, as it turns out, there was a delay in getting hold of the head of operations at our institution. In the meantime, Fedex called me again, saying that it would be possible, using something known as a dummy IEC, the request for which was listed in the above list.

4. Then there was Sunday - a day of rest - and no communications from Fedex.

5. On Monday, there were a flurry of 17 emails. Fedex did not accept my Voter ID as address proof. Only one of the items listed above. And AD code - I thought that since I am not an authorized dealer, I need not submit. But I was wrong. Apparently this is the AD code of the forex section of the bank. Even the bank manager did not know this, and this led to another day of delays.
Expectation - if someone from Fedex had explained this to me, I would have done it immediately.
Reality - I got the same "form" email from Fedex which I received earlier, so I just wrote back saying that I have already submitted the documents. So, this led to a one-day delay in filing the Bill of Entry (BoE) and thus a Rs. 5000 fine. Which has to be paid by me, of course. And even after receiving the AD code, there was some confusion - I got an email saying that the AD code is not valid. I again contacted the bank. The bank said it was valid. Then Fedex confirmed that the AD code was valid, it was the wait for permission to use dummy IEC which was causing the delay.

6. On Tuesday, I got an email with approval list for BoE filing, from jeena.co.in, Fedex India's clearing agents. I filled it up and returned it within 15 minutes. Then came the long wait, 26 Feb to 1 Mar. No intimation from Fedex or Jeena as to what is happening. I tried calling all the numbers available. No one picks up. Calling the toll-free numbers gets me an agent who speaks very soothingly, saying that Mr. (name) is in charge of clearing this package, they will inform him to get in touch with me. But Mr. (name) does not contact me. At all. Ever.
Expectation - if someone from Fedex or Jeena had explained that it would take a week or two for customs to clear it, I would have waited patiently. At the very least, someone should pick up the phone.
Reality - Either the staff are so over-worked and harassed by customers that they cannot bear to pick up calls, or they just stopped caring long ago.

7. On 1st March, I was asked to submit any or all of the following, for value evidence:
1) Internet price
2) Supplier price list
3) Previous BOE,which cleared earlier with same Item
4) If payment is made Bank remittance.

I did so.

8. Again a looong silence. On Mar 7, late in the evening, I get an email, asking for technical writeup. Technical writeup had been submitted in the first batch of documents. Anyway, I again edit that writeup, add a couple of lines, and submit along with all the SLED documentation in the supplier's website.

9. Silence again, till 13 Mar. Now, I am asked to pay the customs duty directly using the ICEGATE website, and to provide "E-way bill". Since I am an individual and not a company, I was not sure if I could do these things. Called up the toll-free. They said they will ask their people to call me and guide me. They never called. I googled, found the ICEGATE website, paid using the details like Challan number supplied by Jeena. But according to various internet sources, individuals need not file E-way bill unless they are transporting high-value goods in their own vehicle. Again write to Fedex, no reply. Again call toll-free. Again they tell me that they will ask their rep to contact me. They do not do so.
Expectation - when the customer asks for help, someone tries to help.
Reality - The toll-free operators either are not allowed to, or not able to, help. The customs clearance people are either overworked or not bothered.

10. 14 Mar - called up Jeena - the executive picked up! That itself was half the battle won. I mentioned that I'm an individual, not a company. Immediately he said, in that case, just send me a letter saying so, no E-way bill required. So, again I sent an email saying just that. And that evening, I get the automated Fedex tracking email, shipment released for delivery.

11. I thought I had paid all that was necessary, but wanted to make sure. Called Fedex toll-free to check. On 15th morning, they said please call back by 15th evening, because some documents may take time to be uploaded into their system. On 15th evening, I call, they say nothing to be paid. On 16th morning, I get a call from Fedex delivery agent, saying I need to pay Rs. 7000 for delivery. Again I call the toll-free number. They tell me that though it is not yet mentioned on their system, it is possible that the clearance charges are what I am expected to pay.
Expectation - clear intimation of what is to be paid, with an emailed invoice in advance.
Reality - confused signals. Nobody knows anything.

12. Part of the problem is the expectation. With services like India Post, you know that there is no one to call to ask for updates, you expect delays, you don't expect any email updates. But then Fedex is more than twice as expensive, so maybe the expectations are warranted. Especially when they put up pages like
http://www.fedex.com/in/why-ship-with-fedex/money-back-guarantee.html
and then completely renege on the delivery date. Like what happened last night - item still not delivered, so delivery status was shifted from "by 8.00 pm" to "pending".

Edit: The item was finally delivered on 20th March, after being shipped on 22 Feb. Even if you discount all the customs clearance delays, it took 8 days. With "Fedex International Priority" shipping. Now compare this with another similar package, sent via Royal Mail and India Post, which took a total of 10 days to reach me from Aberdeen, Scotland.  

Monday, March 18, 2019

different kinds of hash verifications, and writing boot disk iso to USB

Fedora - the hash is in a file called NameofISO-CHECKSUM:
sha256sum -c *CHECKSUM
OpenSuse is similar - the hash is in a file called NameofISO.iso.sha256:

Ubuntu - has sha sums as well as  md5sums, so can use
md5sum -c MD5SUMS 
also.

Then, writing - for Ubuntu based distros, can use the "make startup disk" tool. If the iso is from a different distro, and it is known to be compatible with usb boot, then something like (after verifying the device name - sdd or whatever - very important!)

sudo umount /dev/sdd*
sudo dd if=Fedora-Workstation-Live-x86_64-24-1.2.iso of=/dev/sdd bs=8M status=progress oflag=direct

The above method is known to work for Fedora 24 to 29, and OpenSuse 13.2.

If it is a DVD iso and not meant for USB boot, maybe unetbootin is required. 

Friday, March 15, 2019

burning in subtitles with VLC

From https://www.youtube.com/watch?v=G0NzMlhnzQQ -

Burning in subtitles with VLC needs a bit of tweaking -

1. Media -> Stream, choose the video file. Don't enable the subtitle checkbox on that screen. Click the Stream button.
2. In the Stream Media Wizard, choose the destination as File. Click Add and choose a filename for the destination.
3. In the next Wizard screen, make sure activate transcoding is checked. Click on the settings button, in the subtitles tab, make sure the 'enable subtitles', and 'overlay subtitles on the video' are checked. Codec can remain T.140.
4. In the next screen, in the Generated Stream Output String, we need to edit out (delete) the part of the string which says scodec=t140,
5. Click stream to transcode. 

Saturday, March 02, 2019

F&D smart TV - removing the ads from home screen

We wanted the F&D smart TVs in our lobby to start up with "last selected input" instead of the Home screen with all the youtube "recommendations" or ads. I tried out
Setting -> Common -> System Recovery

(To be updated.)

For Sony TVs, https://www.sony.com/electronics/support/articles/00257833 - Quick settings - Power on behaviour - Last input

There are many posts about using third-party home screen apps etc, but I'd rather use "Last input". Will check on this later.