Tuesday, January 24, 2012

configuring voxel server for sgh and h2h - part 3

Continuing my series of posts on server setup.

Wanted the home page to be not cached, since they update it often. Looking into it, found two interesting articles, at stackoverflow and metaskills. The first article gave a tip on how to control browser cache for specific urls - using the LocationMatch tag. The second one recommended expiring all html pages immediately. I followed the second one, since our server is not facing any resource crunch right now.

ExpiresActive On 
ExpiresDefault "access plus 1 seconds" 
ExpiresByType text/html "access plus 1 seconds" 
ExpiresByType image/gif "access plus 120 minutes" 
ExpiresByType image/jpeg "access plus 120 minutes" 
ExpiresByType image/png "access plus 120 minutes" 
ExpiresByType text/css "access plus 60 minutes" 
ExpiresByType text/javascript "access plus 60 minutes" ExpiresByType application/x-javascript "access plus 60 minutes" ExpiresByType text/xml "access plus 60 minutes"
Initially this block in apache2.conf was inside an if tag, IfModule mod_expires.c - but was not working. Then I commented out the IfModule tags - then it started working, since anyway I had enabled the module using a2enmod expires

Next - wanted to enable some sort of automated log parsing. Among free log analyzers, settled on awstats, since someone had made a mod for wowza too. Followed the ubuntu documentation for awstats installation. There, the recommended cron is

0 */3 * * * /usr/lib/cgi-bin/awstats.pl -config=yourdomain.ext -update > /dev/null

This notation */3 means once every three hours.

For installing GeoIP - tried enabling it directly, since it is supposed to be one of the default included modules, but got this error,

Error: Plugin load for plugin 'geoipfree' failed with return code: Error: Can't locate Geo/IPfree.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . /usr/share/awstats/lib /usr/share/awstats/plugins /usr/lib/cgi-bin/plugins) at (eval 9) line 1.

So now I knew where the plugin directory was - so, could follow the method given here for installing maxmind's geolite databases.

wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
wget http://geolite.maxmind.com/download/geoip/database/asnum/GeoIPASNum.dat.gz

gunzip GeoIP.dat.gz
gunzip GeoLiteCity.dat.gz
gunzip GeoIPASNum.dat.gz

mkdir /usr/local/share/GeoIP/
mv *.dat /usr/local/share/GeoIP/


Tried installing Geo::IP Perl module with
http://www.cpan.org/modules/INSTALL.html
cpan App::cpanminus
cpanm Geo::IP
 FAIL Configure failed for Geo-IP-1.40. See /root/.cpanm/build.log for details

So, followed the steps at
http://antezeta.com/awstats/geoip.html
to install the C module.

wget http://geolite.maxmind.com/download/geoip/api/c/GeoIP.tar.gz
gunzip GeoIP.tar.gz
tar xvf GeoIP.tar.gz
 cd GeoIP-1.4.8
./configure
make
make install


Then,
cpanm Geo::IP worked.

Uncommented
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
changed the path to /usr/local/share/GeoIP/GeoIP.dat

Did not add the other two plugins, for city and ISP, since that would slow down AWstats.


Had to flush newer records, by deleting files in 
DataDir - /var/lib/awstats
in order to process older log files, as given at
http://awstats.sourceforge.net/docs/awstats_faq.html#OLDLOG
Took around 15 minutes to crunch old logs for 15 days, so probably the log for 3 hours will take only half a minute or less.


Next for Wowza. Did not have to modify the logging parameters as given at
http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/
since most of the parameters were already set. But had to change the LogFormat drastically, since my Wowza version is different - 3.03. Also, one point which he makes is to log only the destroy stream event. Not doing that, logging everything as we are doing, was causing over-counting of bytes sent. So, changed the LogFile parameter to read
LogFile=”/usr/local/awstats-wmc/tools/logresolvemerge.pl /usr/local/WowzaMediaServer/logs/*access* | grep destroy |” 


And my LogFormat parameter is
LogFormat="date time %other %other %other %other %code %other %other %other %virtualname %other %other %other %other %url c-ip %method %referer %ua %other %other sc-bytes %other %other %other %other %other %other %other %other %other %other %other %other %other %other %other" 


Using this method, bandwidth is slightly under-reported - by some 10%. But cheaper than Sawmill.


To make pdf reports, on the server, the buildstaticpages tool needed htmldoc, which I installed with apt-get install htmldoc

No comments:

Post a Comment