SERVER UPGRADE - LOG (Feb 2006) - from PB

*********contributed by PB**********************

krishna.radiosai.org (REDHAT 9) to krishna.radiosai.org (MANDRIVA 2006)

Why the shift?

Required new versions of mysql and php to implement the mail list (Phplist) to work.

Possible solutions:

1. Upgrade the mysql and php on the old server.

This had upgrade problems on the old server

2. Configure a new server and forward the request to the page through url redirect.

This was best if the directed url was a public ip.

3. Configure a new server and forward the request to the page through reverse proxy.

This had problems in installing mod_proxy_html on the old server. It rewrites HTML links into a proxy's address space i.e. href pointing to ./files would resolve to http://proxy_address/files.

4. Upgrade the server.

Finally choose to upgrade.

Major Changes:

Old Server	New Server
mysql-server-3.23.54a-11.i386.rpm	mysql-4.1.12-3mdk
php-4.2.2-17.i386.rpm	apache-mod_php-2.0.54_5.0.4-4mdk
httpd-2.0.40-21.i386.rpm	apache-conf-2.0.54-12mdk
postfix-1.1.11-11.i386.rpm	postfix-2.2.5-7mdk
vsftpd-1.1.3-8.i386.rpm	vsftpd-2.0.3-1mdk

Installation Steps:

Got a system having the same hardware configuration as the old one (except the display and ethernet cards)

Chose Mandriva 2006 DVD installation

Partitioned the drive (80GB Drive)

filesystem	Mounted On	Size
/dev/hda1	/	3.7 GB
/dev/hda3	/var	9.2 GB
/dev/hda4	/usr	9.2 GB
/dev/hda6	/tmp	942 MB
/dev/hda7	/home	50 GB

After partition, installation asks "You have the option to copy the contents of dvd to the hard drive before installation. It will then continue from the hard drive and the packages will remain available once the system is fully installed." (Check - copy the whole DVD checkbox)

After installation of required packages, added few other packages that were not in the DVD.

vsftpd-2.0.3-1mdk.i586.rpm (ftp server that supports virtual users) db3-utils-3.3.11-19mdk.i586.rpm (required for vsftpd password generation) libdb3.3-3.3.11-19mdk.i586.rpm (required for db3-utils)
mysql-administrator-1.1.6-linux-i386.tar.gz phplist-2.10.2 (Mailing list server) lame (required for automatic mp3 encoding) librrd (required for shoutstats)

Did a Mandriva update through the Mandriva Control Center (command: mcc)

Transferred required files from the old server

sgh, office, root, webroot
.ssh folders
crontabs (root and sgh)
postfix configuration files (needed slight modificaiton to point correct installation path)
dump of database records from the old server
firewall iptables configuration files
vsftpd (copy all the required files) db3-utils was required here.
icecast server files

Configured the mysql databases through mysql-administration (/root/mysql-administration/bin/mysql-administration)

Restored the dump of database records from the old server.

Ran Nessus to check for security holes. Fixed the following:

ServerTokens Prod (/etc/httpd/conf/httpd.conf) uninstalled mailman

Changed the hardisk in the old server to the new one.

Configured the network.

Enabled ipv4 packet forwarding using the Mandrake Control Center (cmd: mcc) Network Sharing.
This enables ip masquerading. The file /proc/sys/net/ipv4/ip_forward should be set to 1 for this to happen.

Uninstalled shorewall (mandrake firewall)

Restarted network and iptables

Installed nmap and ethereal

Stopped/started services on boot (through Mandrake Control Center).

Present disk stats

Mounted On	Size	Free
/	3.7 GB	3.2 GB
/var	9.2 GB	6.6 GB
/usr	9.2 GB	6.6 GB
/tmp	942 MB	878 MB
/home	50 GB	31 GB

Things to do:

Phplist (already running) to be customized, fine tuned and tested.
Take backup of files ( towrite24kbps to cds)
Write scripts to show the disk stats and database backup
Check for updates monthly (security updates, bug fixes, normal updates)
Clear hardisk regularly (office/sgh)
Automate the start of icecast on boot.
Install nessus

Things to remember:

Icecast location - /usr/local/icecast/bin/icecastmysql-administration location - /root/mysql-administrator/bin/mysql-administrator (admin passwd is same as su) Download files location(rpms,tgz,source files) - /root/Download

*********contributed by PB**********************

Bounty!

Yet another problem with running an only Linux shop - Transcend Jetflash mem sticks. Don't mount on Linux due to some sort of super-floppy partition, it seems. This problem has been around for quite a few years now. Anybody ready to fix it? I can give a 128 MB Transcend Jetflash mem stick + equivalent amount in cash as bounty!

backup script

Made some simple backup scripts for the Postgres database with

pg_dump sgh > sghdbdump

and an sftp script in cron to copy it to krishna.

The dump file is plain text and currently around 50 MB.

For some reason, the method given in this earlier post for automatic ssh login is not working for another account in krishna. I wonder why. Somewhere some typo or something?

sssihl.edu.in mail problem solved

It was because of external hosting, I suppose - mailboxes were not working, but forwarding works fine. So, most probably, that will be what we will do. (With reference to this post...)

places to upload

Interesting link from A - list of places to upload files like rapidshare.de - the link is
http://digital-binary-review.blogspot.com/2006/02/where-to-upload-your-files.html

mail server problems

spent a lot of time trying to configure mail for sssihl.edu.in with 1and1.com - the user interface is somewhat restrictive. Have configured the mx record to point to the records given by
host -t MX sssihl.edu.in ns41.1and1.com

(also host -a sssihl.edu.in ns41.1and1.com)

so the mail servers are mx00.1and1.com and mx01. Pointing from Plesk to these mail servers, still mails to the account I created are bouncing with 550 won't relay to xxxxx@sssihl.edu.in

Maybe the mail servers need time?

Software KM switch?

PB sent me this nice link to Synergy - To share a single mouse and keyboard between multiple computers with different operating systems. May have been useful in the days of working on Dharma and Saispace, but now with Saispace2 most of the time, not so relevant to me. But interesting concept.

PB is changing over krishna.radiosai.org to a new machine, migration work going on. Checked the new one with Nessus yesterday.

LiveSupport

Interesting link sent in by PB - LiveSupport.

Runs a radio station in a way similar to ours, but with the SW running on the playout server also. Searches etc are based on mp3 metadata in the files.

AutoGK's niche

Finally AutoGK finds its niche - where there are read errors, tho DVDDecrypter can make iso files, FairUse Wizard hangs at the point where there are CRC errors. AutoGK carries on just fine, using the IFO mode and extracting to VOB files with DVDDecrypter.

Podcast solution with asp?

This script seems a good solution for audio on demand with downloading instead of streaming.

Fair Use Wizard seems to have a bug? Split Chapters

Fair Use Wizard seems to have a bug: a DVD with a single program chain of duration 2:08 with three chapters of roughly 40 minutes each, when ripped with the option 'Split chapters' and total filesize 540 MB, gave 2 files of 180 MB and one of 70 MB, and also the 3 files were of 37 minutes duration! Doing the rip without the 'Split chapters' option => proper file, no more problems.

Fair Use Wiz timing

Seeing its logs, Fair Use Wizard has taken roughly 1.5 times real time for approx 700 kbits per sec video encoding with the internal XvidD codec. From ISO file. The indexing was done earlier, approx 5 min per 2 hour DVD.

http://www.drs2006.com/manuals/faq/plsasx.html gives info about pls and asx files in one place. But the pls files' titles are still not being reflected on Xmms, and the filenames are being shown. Probably since Xmms doesn't support Version 2 pls files? Maybe I have to tweak the server settings to not show the filenames?

Server being a relay automatically shows the origin server's title setting. If it is not a relay? Have to try out and see locally.

asx files working with shoutcast

Finally! asx playlists for shoutcast on demand files are working even for mp3s with spaces in filename.

The actual problem was not that the filename had spaces in it, but WMplayer can't play mp3s of size below 4 seconds! Tests with larger files are working.

added counter

Added a counter to this, from Webstats4U

for printing mails

Or in general for exporting mails from KMail - make a folder, make it an mbox folder, copy the required mails to it. If the folder is made in the top level, it is seen directly in ~/.Mail

Eudora can open the mbox file directly as it is its native format - just copy it to Eudora's Mail folder.

Or use MHonArc to convert to html first.

Just an aside - tho these posts only talk about the exciting stuff, most of the time is spent in audio processing, making playlists etc...

changed line in /etc/sysconfig/clock (in Mandrake 10) to read

UTC=false

so that the clock shows correct time in IST in XP (dual boot) also.

easier way to make Windows Media Player play shoutcast streams

In case you can host your own files and just need to point those with Windows Media players to a shoutcast stream, making an asx file works:

and the syntax of the asx file is simply

<asx version="3.0">
<entry>
<ref href="http://stream.radiosai.org:8000/">
</entry>
</asx>

Unfortunately I've lost the link which showed multiple files.
Now when I try these asx files with links to files
(Shoutcast streaming on demand)
it works with filenames with no spaces,
but not for filenames with spaces...

Experimenting with audio on demand

Both Shoutcast and Icecast do audio on demand. Shoutcast even makes a pls file on the fly. But we would like to have our own pls files, with more than one file in the playlist, since we would like to have the announcement and then the programme. Following the pls file format,
[playlist]
NumberOfEntries=2
File1=http://stream.radiosai.org:8000/content/file1.mp3
File2=http://stream.radiosai.org:8000/content/file2.mp3

Is working. But content has to be a folder, can't be a shortcut. What about soft links and hard links? Googling
found this google answer
and searching sysinternals got junction


And that works!

Does not allow downloading using right-click and save-as with browsers, either.

But playing in Windows Media Player? Googling
gave this nice link.

But it has two problems: Plays only the first file for files without spaces in them, (of course, that's what it's coded for!) and doesn't play files with spaces (%20) in them since WMP converts the %20 to spaces!

Tried from command-line with
wmplayer "filename with spaces.mp3"
wmplayer "filename%20with%20spaces.mp3"
wmplayer filenamewithoutspaces.mp3
wmplayer "filenamewithoutspaces.mp3"

Only the last two work.

Provisional requirements for Web Schedule display

1. Display earlier days' schedule using clickable calendar - so, having a MySQL backend instead of the current Access mdb file makes sense - otherwise the mdb file will become bigger and bigger...
2. Update with an uploadable csv file, automagically delete entries older than a month old (if necessary).
3. Look and feel of current site, but tables could look better.


Other fancy ideas
1. Linking to audio on demand?? Ideas on implementing??
2. Automagically linking to relevant Prasanthi Bulletin page
3. Better way of presenting the categories than the current method?
4. Does the BBC look good?

Hyperscore

Wired News has an interesting story about Hyperscore - interesting midi creation tool. Free version can make songs 30 sec long for use as ringtones etc. Of course, when we have Yamahas and Korgs with midi cables and skilled players, why bother....