In order to automate some of our processes, wanted a PHP page to trigger some file copy commands to a shared directory. Initially the network share was not writable by the web server.
I have made it writableby mounting it with file_mode=0777,dir_mode=0777
After upgrading my machine from Linux Mint 20.3 to 21, print job notifications were making "ping" sounds.
Opened the "Sound" configuration applet, Sounds tab - turned off the sound for "Showing notifications" after scrolling down. This seems to be a newly added sound, since earlier I had turned off all sounds. (At first glance, I didn't see it, but it was visible after scrolling down.)
Saw this error in the logs when trying to start VNC on a machine where VNC was earlier working, but was now displaying a blank screen.
https://superuser.com/questions/1607082/tightvnc-in-ubuntu
advises to
Change the ~/.vnc/xstartup file to:
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
startxfce4 &
This worked - the session commands were not present before.
There seemed to be something wrong with win-acme which was preventing automatic renewals of the LetsEncrypt certificate on one of our servers. Possible reason could be that the server is switched off during the nights as it is a test / development server. Anyway, wanted to try Cloudflare's Origin server certificate instead, like we'd done for a Linux bitnami server earlier.
This post
https://bytefreaks.net/applications/cloudflare-origin-server-certificate-for-iis-10-server-on-windows-server-2016-to-allow-full-strict-mode-ssl-tls-encryption-mode
gives a good step-by-step guide. (Archived link)
which is perhaps less clear.
When I try to select the cloudflare cert, the following error message is displayed:
I suppose that is because the certificate is from Cloudflare and not from one of the recognized certificate providers. Works with Cloudflare proxying in Full/ Strict mode. As the documentation says, "Site visitors may see untrusted certificate errors if you pause or disable Cloudflare on subdomains that use Origin CA certificates. These certificates only encrypt traffic between Cloudflare and your origin server, not traffic from client browsers to your origin."
1. Made ourdomain.tld and www.ourdomain.tld proxied through cloudflare, from cloudflare dashboard - this is required for the next two points,
2. Added a page rule to redirect all requests to the bare domain ourdomain.tld to www.ourdomain.tld (which points to the google sites website).
3. Verified that "Always use HTTPS" is turned on in cloudflare Zone settings --> SSL/TLS --> Edge Certificates page. This will ensure that any http requests will be redirected to https.
Googling brought up PosteRazor - a nifty tool - which has an online webassembly-based version also. Worked perfectly in Firefox even on my underpowered machine with only 4 GB RAM running Linux Mint. Embedding their how-to video below -
This is useful when you want to load a list of files or something like that when the user click on a link, using Ajax - How To Make a Modal Box With CSS and JavaScript
https://www.educative.io/answers/how-to-get-keys-values-and-entries-in-javascript-object
If we don't know what a javascript object contains, we can use these to learn more about the object -
Object.keys(obj) – returns all the keys of object as array
Object.values(obj) – returns all the values of the object as array
Object.entries(obj) – returns an array of [key, value]
and we can check the length of the object using Object.keys(obj).length
using which we can loop through the object, or we can use foreach or for var in array and so on.
messages.forEach(function(message) {
console.log(message);
}
for(let message of messages){
console.log(message);
}
for(let key in messages){
console.log(messages[key]);
}
data.messages.forEach((obj, i) => {
console.log("msgFrom", obj.msgFrom);
console.log("msgBody", obj.msgBody);
});
data.messages.map((obj, i) => {
console.log("msgFrom", obj.msgFrom);
console.log("msgBody", obj.msgBody);
});
We can also implement lookup tables using objects,
https://www.educative.io/answers/how-to-use-objects-for-lookup-in-javascript
To add to my earlier post about removing files from a github repository, and from git history, github's documentation is at
python3 -m pip install --user git-filter-repo
HFS plus formatted Mac removable SSD (USB external hard disk) was not recognized on Windows and not writable on Linux.
perhaps can be force mounted?
https://forums.linuxmint.com/viewtopic.php?t=283956
Or perhaps would need journaling turned off?
https://help.ubuntu.com/community/hfsplus
Anyway, with journaling turned off, exFAT is more compatible.
Trying to format it to exFAT on Linux -
https://unix.stackexchange.com/questions/61209/create-and-format-exfat-partition-from-linux
on my system,
sudo apt-get install exfat-utils exfat-fuse
(all installed already)
and then gparted
But greyed out in gparted - apparently a newer version of gparted is needed - my gparted version was 1.0 - so used Disks instead, OK.
But did not read on Windows. Reformatted in Windows, then OK. Some blocks issue, I guess. Or maybe needs to be GPT? I've not yet tried on Mac.
ffmpeg -framerate $FRAMERATE -i "image%05d.jpg" output.mp4
and if we want XVID,
https://trac.ffmpeg.org/wiki/Encode/MPEG-4
ffmpeg -framerate $FRAMERATE -i "image%05d.jpg" -c:v mpeg4 -vtag xvid output.avi
(not yet tested for compatibility with our Blu Ray player.)
What I've used for 4K frames,
ffmpeg -r 30 -f image2 -i "paranal_timelapse_vst_%05d.jpg" -vcodec libx264 -crf 15 -pix_fmt yuv420p paranal_timelapse_vst-4k-frames.mp4
Edit:
# for starting at arbitrary frame number
ffmpeg -r 30 -start_number 405 -f image2 -i "paranal_timelapse_vst_%05d.jpg" -vcodec libx264 -crf 15 -pix_fmt yuv420p partial.mp4
Apparently in Adobe Acrobat, it is by Edit PDF -> Link -> Remove web links
But then if the PDF is viewed inside Moodle, Moodle puts the web links back by parsing the text? Maybe. Works OK for a normal webserver.
var mytable = document.getElementById("var queryString = location.search
let params = new URLSearchParams(queryString)
// example of retrieving 'id' parameter
let id = parseInt(params.get("id"))
console.log(id)Following https://mehulgohil.com/blog/how-to-export-data-to-csv-using-php/
In our case, the required code was something like this:
header('Content-Type: text/csv; charset=utf-8');
header('Content-Disposition: attachment; filename=filemastercsv_export.csv');
require("SessionCheck.php");
require("Constants.php");
$output = fopen( 'php://output', 'w' );
ob_end_clean();
$query= "select * from our_table_master f where f.fileId > 50900 order by f.fileId";
$dbcon= mysql_connect($DBHOST,$DBUSER,$DBPASSWORD);
if(!$dbcon)
{
echo "Unable to connect to database !";
return;
}
mysql_select_db($DBNAME);
$totalrows= 0;
$result= mysql_query($query);
if (mysql_num_rows($result) == 0) {
echo "No rows found, nothing to print so am exiting";
exit;
}
while ($row = mysql_fetch_row($result)) {
fputcsv($output, $row);
}
mysql_free_result($result);
mysql_close($dbcon);
exit;
A link dump on populating a Google Apps html template, followed by the javascript solution (without Google Apps) which was finally used.
https://developers.google.com/
explanation of problem
Passing data to html service - Desktop Liberation (google.com)
apparently safe way to do it
One of our pages which had an embedded twitter timeline widget had the issue that the twitter widget was not loading on Microsoft Edge browser, but would load on Firefox / Chromium. Looking at the console (right-click on the page, Inspect --> Console tab), found that Edge was reporting "Tracking prevention blocked a script resource from loading", along with a Bing search link.
One disadvantage of using the typewriter effect in javascript instead of a spinner gif is that we can't replace the text with something else when finished -
$(#"divname").hide() does not seem to work.
What does seem to work is
document.getElementById('typingdiv').innerHTML = '<br><br>'
So we can use that with either a gif created with a "typing gif creator" like msgif.net or with a typewriter effect in css as the initial innerHTML.
Logging in to Google Cloud Shell, noted the "help" options, and a useful function -if we have some file on our home directory in google cloud shell and want to download it, we need not struggle with setting up ssh keys and so on - an easy way is to use
dl filename
in cloud console - this means download to your client machine (browser). To download a file into cloud shell, we can use standard
wget url
etc.
Since we need to use client-side Ajax to update html pages of the Google Apps Script Web app - https://developers.google.com/apps-script/guides/html/best-practices - one way to get a terminal-like or console log-like experience in the web app would be like this -
In the code.gs,
In the Page.html, a <script> section at the beginning, which has our ajax code, and the <body> section containing the suitable formatted blocks with the relevant ids as called by the ajax functions.
The method mentioned at https://stackoverflow.com/questions/20256760/javascript-console-log-to-html for redirecting console.log did not work for me, maybe because console.log logs to the console of the client making the call, and in our case, the relevant functions are server-side. Or maybe there are other differences, like the (function () { }) call syntax seems to be different from the one below:
Navigating the new analytics.google.com - takes a bit of getting used to.
Some interesting things about client-side javascript date comparison -
(The following is a link dump of various possibilities searched out for implementing this, for which plain javascript without Google Apps script was finally used.)
Link dump:
We can automatically choose the user's timezone, and also have a timezone setting from any of these,
console.log(Intl.DateTimeForma
Documentation
https://developer.mozilla.org/ This post talks about manually downloading and uploading, using Google Apps Script only to generate the webpage once and save it to Drive.
https://stackoverflow.com/questions/70071534/i-want-to-use-google-app-script-with-html-service-to-create-a-static-html-file
I suppose the file saved to Drive can be uploaded to some other site using some sort of scripting running either on Google Apps Script or on our local server.
Edit: Possible uploading to S3 bucket using Google Apps Script code similar to this.
Edit Mar 2023: Implemented a Github Pages upload version using Github's REST API in the PHP translation of our Google Apps Script mentioned in a later post.
- a good reference for what details are needed, and where what needs to be filled up, for those with Capital gains from Mutual Funds.
Edit: June 17 -
Apparently it's not a good idea to file before June 15 - https://www.livemint.com/money/personal-finance/why-you-should-avoid-filing-tax-returns-before-15-june-11686674063937.html
And while there is the option of uploading a csv file, in my case, the manual entry of the data needed for each Mutual Fund transaction was the easier way, since some of the data was not directly 1:1 matching in the format the IT department wanted vs the format provided by Kfintech / CAMS in their profit/loss statement. For example, the ASIN information was part of the "fund name" field, we need to copy-paste it out (or write code to check for it and take it out, which would make the process more complex).
And also found that Groww (and other such online mutual fund facilitators) provides only profit/loss from the funds purchased through Groww - Kfintech + CAMS statements are needed for the full picture.
Also, for ease of opening the protected Excel sheets with Google Sheets, the method I used was to download them, open them locally using LibreOffice, then save as .ods or something like that without password protection, and then upload to Google sheets.
jquery append() actually appends an element. So, we need to make the append statement an entire row, in case we're appending to a table.
Populate html table on jQuery success event - Stack Overflow
table.append("<tr><td>"+elem.username+"</td><td>"+elem.name+"</td> <td>"+elem.lastname+"</td></tr>");
and not something like
table.append("<tr><td>");table.append(elem.username);table.append("</td><td>");etc.
Doing a table fill with AJAX, I was trying to use $('#maintable').append() multiple times in a loop, assuming that the append function is just appending whatever text I supply to it - but it turns out jQuery append actually appends a child element to the parent element. The above syntax is for jQuery and not plain javascript.
I was ending up with content like
<tr></tr>
<tr></tr>
<tr></tr>
<td>Our data</td> ... and so on.
So, I must append an entire row with a single append statement. In javascript, the function call can span multiple lines, no problem. Most probably the correct way would be to use maintable.tbody.append, though the snippet below works.
function showThings(things) {
| var maintable = $('#maintable'); |
| maintable.empty(); |
| |
| for (var i = 0; i < things.length; i++) { |
| maintable.append('<tr><td>Row '+ i + '</td><td>' + things[i][0] + '</td><td>' + |
| things[i][1] + '</td><td>' + |
| things[i][2] + '</td><td>' + |
| things[i][3] + '</td><td>' + |
| things[i][4] + '</td><td>' + |
| things[i].length + '</td></tr>'); |
| |
| } |
| |
Adapted from an email from PB:
Executed the following SQL in the local postgresql database:
UPDATE file_information SET description = REPLACE(description,'http://old.domain/', 'https://new.domain/sss/') where description like '%old.domain%'
1539 row(s) affected.
UPDATE file_information SET description = REPLACE(description,'http://www.old.domain/', 'https://new.domain/sss/') where description like '%old.domain%'
354 row(s) affected.
Executed the following SQL on the web server:
UPDATE `our_file_master` SET `description` = REPLACE(description,'http://old.domain/', 'https://new.domain/sss/') where description like '%old.domain%'
(1022 rows)
and
UPDATE `our_file_master` SET `description` = REPLACE(description,'http://www.old.domain/', 'https://new.domain/sss/') where description like '%old.domain%'
(345 rows)
I wanted to export quite a large subset of a table which was using up 1.8 GB - first tried this method,
https://phoenixnap.com/kb/
SELECT ourfield1, ourfield2, ourfield3 FROM ourTable
INTO OUTFILE '/path/to/ExportFile.csv'
FIELDS ENCLOSED BY '"'
TERMINATED BY ';'
ESCAPED BY '"'
LINES TERMINATED BY '\r\n';
This failed, --secure-file-priv is set to null(?) on our server and I didn't want to change that.
database - How should I resolve --secure-file-priv in MySQL? - Stack Overflow
Next plan was to export by redirecting a here document in a bash script. One hiccup was that the here doc should end with the end characters without any indent, and there should be no other characters after it on its line, otherwise the script fails to find the characters.
https://stackoverflow.com/
Next issue was that the exported data was large, and probably not indexed, so
mysql: Out of memory (Needed 698144116 bytes)
For this, I would probably need to limit the number of result rows (by timestamp, in this case),
How do I get the current Unix time in milliseconds in Bash? - Server Fault
How to increment a variable in bash? - Ask Ubuntu
Then, the easiest way to code seemed to be to put the entire statement in one line, avoiding issues with SQL/bash multi-line statements:
$MYSQL --batch -u $MyUSER -h $MyHOST -p$MyPASS $DBS -e "select l.id, l.timecreated, l.userid, l.ip, f.filename from prefix_logstore_standard_log l left join prefix_files f on l.contextid = f.contextid where l.timecreated > $StartDate and l.timecreated < $EndDate and l.action = 'viewed' and l.target='course_module' and f.filename like '%.pp%' order by l.timecreated;" > $FILE
This would actually give a tab separated file and not comma separated, but this is also imported without issues by Google Sheets.
Checking for open ports on one of our servers using netcat,
nc -zv our.server.name 2000
Connection to our.server.name 2000 port [tcp/cisco-sccp] succeeded!
Googling, found that this could be due to our ISP trying to block VOIP ports.