Saturday, October 28, 2023

moodle site registration issue and fix

There was a problem with site registration, saying "your site is already registered". This is probably because we had copy-pasted the database from an existing site. 
 
This discussion 
suggests deleting the entry in the registration_hubs table in the database to allow us to register.

I have done that now, and that allowed the site to be registered.

request for new font in Moodle instance

Following a request to install a new font in a Moodle instance, I did a local install with

sudo mkdir -p /user/share/fonts/googlefonts 
cd /usr/share/fonts/googlefonts
sudo unzip -d . ~/Tangerine.zip
sudo fc-cache -fv
Links - Using web fonts -
Using local installation -

 
But then it was reported that this font is not seen as one of the options when editing an element in the custom certificate plugin.
 
Certificate builder needs fonts in a separate format?
 
Tried purging cache, purging cloudflare cache etc, but still not seen. 
 
Not the manual method, but this plugin method in the up-to-date docs.


Finally, trying the TCPDF fonts plugin
As screenshot shows,
Dashboard - Site Administration - Server - TCPDF Fonts
 
After initialization and adding the font that way, works fine - did not have to purge caches etc.
 
 

Thursday, October 26, 2023

find version and other details of installed package with apt

 Via Check version of Installed Package on Ubuntu / Debian | ComputingForGeeks

sudo apt show nameofpackage

more reading about prototype pollution - even front-end can have vulnerabilities

Checking to see how critical are prototype pollution vulnerabilities for front-end code, apparently they are:

javascript - Is prototype pollution only exploitable on the back end? - Information Security Stack Exchange


google photos links sent by email, not working on iPhone

I had shared some google photos albums using the sharing link, which is of the form https://photos.app.goo.gl/<id> and A said that they could not see the album. 

My reply was, 

those links are probably not opening in your iPhone because your phone's Apple Mail is not configured to use Safari or Chrome browser to open links.

It might work if you long press on the link, choose "Copy", and paste the link in Safari or Chrome browser in your phone.
And that seemed to work.

Mongodb pros and cons

One of the many sites which list Mongodb vs Postgres comparisons, 

https://www.knowledgenile.com/blogs/pros-and-cons-of-mongodb#DisadvantagesofMongoDB

Transaction-safety and joins etc are what mongodb doesn't offer, while sharding is its strength.

Wednesday, October 25, 2023

Amazon firestick not recognizing a USB drive - issue was with partitioning

There was one drive which was being recognized, and another drive which was not. The drive which was being recognized had the original 16 GB FAT32 partition, while the one which was not being recognized had been used as a Linux Mint installation disk, and had been later reformatted as FAT32 using the "Disks" utility on Linux Mint.

Both show in "Disks" on Linux Mint the following info:
 
 

Suspecting that block size might be the issue, to show the block size,

 
stat -f shows  8192 for the non-working drive and 16384 for the working drive. Tried to see if reformatting would work -

(trying 32k size instead of 16k)
 sudo mkfs.fat /dev/(device name from above) -s 64 -F 16
 

Changed label using
Disks --> Edit Filesystem --> it prompts for the label.
 
But that didn't seem to work - copying a test file seemed to take ages. 

Then, tried GParted. GParted showed the partition to be of type iso9660! 

Unmounted, Deleted the partition, recreated partition table with device -> create partition table (msdos), formatted as msdos, then the Firestick recognizes it OK even though the block size is 8K as shown in 

 stat -f /media/mac/LM
  File: "/media/mac/LM"
    ID: 83100000000 Namelen: 1530    Type: msdos
Block size: 8192       Fundamental block size: 8192
Blocks: Total: 1951116    Free: 1318805    Available: 1318805
Inodes: Total: 0          Free: 0


 

Sunday, October 22, 2023

upload artifact from github actions build

Create a Release and Upload Artifacts with GitHub Actions | Thomas Stringer (trstringer.com)


Easy way to publish artifacts - 

https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/How-to-publish-GitHub-Actions-artifacts-example

Just add the following, and it will create a zip file with the path -

- name: Upload Artifact
      uses: actions/upload-artifact@v2
      with:
        name: artifact-of-build
        path: platforms/android/app/build/outputs/

Saturday, October 21, 2023

optimizing github actions build using cache

When I was poking around the github actions tab, saw the cache side-bar link, and thought I'd check it out. 

When added to this msbuild action, the OpenCV caching brought down the total build time from 5-8 minutes to just 1.5 minutes - the download and untar from cache happens in 25 seconds instead of the download and unzip from github releases which would take 6 minutes or so.

More references:

Need to order the named jobs so that the cache comes first - https://github.com/actions/cache#skipping-steps-based-on-cache-hit

Wednesday, October 18, 2023

Travis deploy to Github releases - requires classic personal access token

OCVWarp deployment from Travis-CI to Github Releases failed because the token had expired. When creating a new token, I tried a fine-grained token with write permissions to releases on that repo alone. But that did not seem to work. A "classic" personal token with "repo" scope worked fine. 

https://github.com/settings/tokens

Sunday, October 15, 2023

migrating from Android to iPhone


How to migrate your WhatsApp data from Android to iPhone | WhatsApp Help Center

 Though this method was followed, only the photos and media were transferred from whatsapp, not the chat history. Probably that was because whatsapp had been earlier installed on the iphone, though it was then removed with "delete all data"?

https://faq.whatsapp.com/1195710084347483?helpref=faq_content

then, to transfer music, move content manually from your Android device to your iPhone,

https://support.apple.com/en-gb/HT205063

Friday, October 13, 2023

playing local video from firestick without internet

While setting up the Firestick, we wanted to create a new dedicated account for it, but it asked for a unique phone number - so we just used my account.

We found VLC player was stuttering with our 4K videos on USB, but MXPlayer works with hardware acceleration, and plays them perfectly. To access MXPlayer without internet, via Quora,

Settings --> “Applications” --> “Manage installed applications” --> "MXPlayer" --> "Launch"

And it looks like there is no "shutdown" option - we just press "home" to come out of MXPlayer, and then turn off power to the device.


Maybe this will? Long press centrebutton + play for 10 sec

Our Firestick runs FireOS 6.4, but we did not have to do any ADB commands to use FAT32 formatted USB memory. The ADB commands in this pdf may be useful for increasing system usable memory, installing apps and so on, but not needed for us.

As recommended in this troubleshooting video, when connecting for the first time, doing so from the "home" screen might be beneficial.

How to play our own content from USB on Amazon Firestick

We can't just connect the firestick to a computer and transfer files to its internal memory - MTP does not work with Firestick 4k

Can probably use adb pull


But far simpler to just use a removable USB drive.


Thursday, October 12, 2023

error updating snapd

Investigating unresponsive server behaviour on one of our servers,

grep -i error /var/log/syslog

showed some lines with WALinuxAgent - this is Azure's agent, I guess, as well as snapd.

 snapd[63805]: stateengine.go:149: state ensure error: cannot query the store for updates

 To check if we can remove snapd


snap list
 
shows lxd, which might be needed - 

How to remove LXD from my system - LXD - Linux Containers Forum

So I left it alone, and tried a manual update with

 sudo snap refresh

which indicated all packages were up to date. So maybe the error was a temporary error.

issue with one of our Linux servers

One of our Linux servers was becoming unresponsive around 2-5 pm daily over 2-3 days. Checked the logs as in my earlier post but did not find anything except the usual SSH brute-forcing attacks. Changed the port from 22 to something else by
sudo nano /etc/ssh/sshd_config
sudo service sshd restart

And opened the new port on Azure portal Networking screen. 

This seems to have worked.

Wednesday, October 11, 2023

redesigning python code to fade out smart lamps

 The smart lamp controls with a python flask app as in a previous post had some issues. 

  1. There was no feedback to the user in case the mobile browser lost connection or the python app lost connection to the smart lamps.

  2. The raspberry pi was showing slightly excessive cpu usage, with the lightning bolt icon on the top right corner appearing sometimes.

  3. There was no way to stop the python loop in case it was taking too long to complete.

  4. The way I'd written it, if the /fadein url was the last used one before mobile browser disconnect, the next time I opened the mobile browser, the lights would come on! due to the mobile browser retrying that page.

So, thought of redesigning the code. 

Found that RaspController has a section for running user-defined custom commands. Since both KR and I use RaspController on our phones, the simplest solution seemed to be to just use a python script to do the fadeout of the lights, and show the output of the script on the terminal using RaspController. 

And that seems to be working fine. 

Since the code on github has the device ids and local keys, instead of taking the time to modify the code to put the private info in separate files etc, I'll just copy-paste the current state of the code here.

import tinytuya, sys
from time import sleep

def moveup(lines):
        for _ in range(lines):
            sys.stdout.write("\x1b[A")

nowait = bool(1);

# Connect to Device
d = tinytuya.BulbDevice(
    dev_id='changeme',
    address='our.ip.2',      # Or set to 'Auto' to auto-discover IP address
    local_key="changeme12",
    version=3.5)
print("Connected to d")
b = tinytuya.BulbDevice(
    dev_id='deviceidchange',
    address='our.ip.1',      # Or set to 'Auto' to auto-discover IP address, but that takes 10 sec
    local_key="changeme34",
    version=3.4)
print("Connected to b")

def fadeout():
    d.set_brightness_percentage(100, nowait)
    sleep(1)
    b.set_brightness_percentage(100, nowait)
    sleep(1)
    d.turn_on()
    print("d turn on")
    sleep(1)
    b.turn_on()
    print("b turn on")
    sleep(1)
    i = 100
    while i > 10:
      d.set_brightness_percentage(i, nowait)
      print("d brightness {0:3d}".format(i))
      b.set_brightness_percentage(i, nowait)
      print("b brightness {0:3d}".format(i), end='\r')
      moveup(1)
      sleep(1)
      i -= 10
    i = 10
    while i > 11:
      d.set_brightness_percentage(i, nowait)
      print("d brightness {0:3d}".format(i))
      b.set_brightness_percentage(i, nowait)
      print("b brightness {0:3d}".format(i), end='\r')
      moveup(1)
      i -= 1
    d.turn_off()
    print("d turned off.          ")
    b.turn_off()
    print("b turned off.          ")

fadeout() 

Edit: unfortunately, this Tuya solution had issues when the internet connection was down - the response to API calls would become extremely slow - maybe taking a minute or two after some sort of timeout - making the fade in/out loop pretty much non-functional. As suggested by one of the volunteers at a sister institution, we tried Wiz lamps and LED strips instead, which work flawlessly without an internet connection using a Wiz remote as mentioned in a newer post.

finding duplicates to fix SQL query - ad hoc query in Moodle

There was an issue once again with the feedback ad-hoc Moodle report - checking the query in DBeaver, the database returns an error "Subquery returns more than one row" for some dates. Eg Oct 1.

Via https://stackoverflow.com/questions/2594829/finding-duplicate-values-in-a-sql-table

Found the duplicate timestamps with

select timemodified, count(*) from ourprefix_feedback_completed
GROUP BY timemodified
HAVING COUNT(*) > 1

1690263769    2
1696119639    5


Have changed the timestamps to increment the duplicates by one, so that the query works without issues.

 

Tuesday, October 10, 2023

blender rendering notes

I tried out some rendering of scenes using the Cycles Panorama camera 180 degree Fisheye Equidistant option. A large number of links to howtos for accomplishing specific tasks in blender - link dump below.

1. Render optimizations - 4K render of one frame takes 6:45 - nearly 7 minutes a frame is not enough - need to check out GPU rendering. GPU needs min compute capability 3 Nvidia or AMD GPU -
https://www.youtube.com/watch?v=1io1yPj0Sk0.
https://docs.blender.org/manual/en/latest/render/cycles/gpu_rendering.html
For reference, the old Macbook Pro with 4 GB RAM and i7 processor has a compute capability of 1.2 -
From System Info
NVIDIA Corporation GT216M [GeForce GT 330M]
GeForce GT 330M compute capability is 1.2
But for tests, can do with some optimizations - render time tweaks -
5 Tips for FASTER Renders in Blender Cycles - YouTube

Noise threshold of 0.5 and enabling noise threshold reduced render times nearly 50%.

2. Animate rocket launch tutorials -
Involved version - https://youtu.be/NFtXTruJ-_c
Simpler version - Rocket Launch Smoke Simulation in Blender 3.0 [Easy!] - YouTube - this is the one I tried out, removing the grass in the foreground and adding water instead.

 
4. Blenderkit - For current versions of blender, need to install blenderkit addon from blenderkit.com
BIG CHANGE! You won't find BlenderKit in Blender 3.0 by default anymore || Blender Tutorial - YouTube

5. The reason the smoke was not working for my rocket was the Bake settings, and the paths to the bake were also set wrongly.  Need to set the paths correctly, Free all, then bake all.
In Physics tab, which is the one which is below the particles tab, with a sphere in orbit around another sphere, 
choosing SmokeDomain object,
under Cache,
changed the directory of the cache to within the project directory,
"Free all"
Then 
"Bake all" - this took approx 3 sec per frame in the beginning, becoming 10 sec per frame near the end.
 
and if only Sky is needed, Nishita sky seems to be a good option - 
 
7. Make all paths relative - so that the project directory can be copied and used on another machine - 
https://blender.stackexchange.com/questions/230328/how-to-package-all-assets-image-textures-mainly-into-a-folder-not-into-blend
File --> External Data --> Make all Paths relative
 
8. Importing a model - like FBX - with textures -
Imported 3D model missing textures - Simple Fix - YouTube - which says just re-apply the textures - Material properties --> Surface --> Emission --> Color--> Image Texture --> open file.
Blender 2.8 - OBJ materials are not imported properly - [ SOLVED ] - YouTube - this one renames the mtl file, if that was the issue.
How to PROPERLY Import Models With Textures in Blender - YouTube - this also uses Node Wrangler, which has a detailed tutorial below - 
What I needed was Node Wrangler:
How to use node wrangler for setting texture roughness, colour etc in one step.
 
9. Moving all keyframes to make a 150 frame video 1500 frames instead -  with the current version of blender's smoke simulator, just doing the time remapping as 10 to 100
https://blender.stackexchange.com/questions/3242/can-i-increase-the-frame-rate-of-an-animation-while-keeping-it-the-same-length
http://www.ffnn.nl/pages/articles/media/blender-tips-and-tricks.php
- is not sufficient. The smoke gets updated only once in 10 frames if the time remapping is 10 to 100. Trying 100 to 1000 - the maximum value for the latter value is 999 - so 100 to 999, but the resulting animation still has the smoke particle system jumping from one state to the next every 9-10 frames. So the only way is to add more frames to the project, and drag the keyframes further along the timeline. A separate post about it coming up.





Monday, October 09, 2023

remove loggly

I'd installed loggly free trial on one of our servers, and had forgotten about it. Installation documentation
https://documentation.solarwinds.com/en/success_center/loggly/content/admin/sending-apache-logs.htm

 When it came to uninstalling it, Solarwinds support said just remove the API token. But then the server would continue to send the data to solarwinds' server, only we can't see it, right? Not very customer-friendly. Anyway, their own documentation says

Stop Sending Data to Loggly (solarwinds.com) 

sudo bash ./configure-linux.sh -a our.domain.org -r
 

for removing the service. So I did that also, and removed the token also.

Sunday, October 08, 2023

simple way to overwrite text in terminal using python

https://stackoverflow.com/questions/4897359/output-to-the-same-line-overwriting-previous-output
 

just add , end='\r' to the print()

So, something like the code in this private repo,
https://github.com/hn-88/python-tuya-fadein-web/

while i > 10:
      d.set_brightness_percentage(i, nowait)
      print("d brightness {0:3d}".format(i))
      b.set_brightness_percentage(i, nowait)
      print("b brightness {0:3d}".format(i), end='\r')
      moveup(1)
      sleep(1)
      i -= 10

Friday, October 06, 2023

converting youtube 360 VR webm files to equirectangular

Earlier, I had downloaded some youtube 360 VR files, which were in equirectangular format, converted to either fulldome or pre-warped mirrordome using OCVWarp, and projected on our dome. For example

360° Video of Vidhan Soudha, Bengaluru During On Going Lock-Down  

But (recently?) when I tried out 360 VR 4K videos like this one

Tim Peake ISS 360° Planetarium Tour

the format was different - not equirectangular. While I thought about exploring the format to find out some workaround, found that Paul Bourke has already done it in Jan 2020.

YouTube 360 video format

Apparently it's 6 faces of the conventional cube map, but resized a bit. Ordered as left-front-right
bottom-back-top

And he mentions the ffmpeg command to convert it to equirectangular, which I tested and seemed to work,

ffmpeg \
-y \
-hide_banner \
-i input.mkv \
-vf "v360=c3x2:e:cubic:in_forder='lfrdbu':in_frot='000313',scale=3840:1920,setsar=1:1" \
-pix_fmt yuv420p -c:v libx264 -preset faster -crf 21 \
-c:a copy -ss 16 -t 10 -movflags +faststart \
output.mp4





Wednesday, October 04, 2023

send a kill command to a process from python

Different methods are mentioned here, https://stackoverflow.com/questions/1489669/how-to-exit-the-entire-application-from-a-python-thread

os._exit is given as one of the replies. "should normally only be used in the child process after a fork()"

But since this was for my smart lamps scripts, did not implement any of this. 

 Edit - Also see my updated post on the python script, which allows us to just hit ctrl-c to interrupt - https://hnsws.blogspot.com/2023/10/redesigning-python-code-to-fade-out.html