Continuing my series of posts on server setup.
Wanted the home page to be not cached, since they update it often. Looking into it, found two interesting articles, at stackoverflow and metaskills. The first article gave a tip on how to control browser cache for specific urls - using the LocationMatch tag. The second one recommended expiring all html pages immediately. I followed the second one, since our server is not facing any resource crunch right now.
ExpiresActive On
Wanted the home page to be not cached, since they update it often. Looking into it, found two interesting articles, at stackoverflow and metaskills. The first article gave a tip on how to control browser cache for specific urls - using the LocationMatch tag. The second one recommended expiring all html pages immediately. I followed the second one, since our server is not facing any resource crunch right now.
ExpiresActive On
ExpiresDefault "access plus 1 seconds"
ExpiresByType text/html "access plus 1 seconds"
ExpiresByType image/gif "access plus 120 minutes"
ExpiresByType image/jpeg "access plus 120 minutes"
ExpiresByType image/png "access plus 120 minutes"
ExpiresByType text/css "access plus 60 minutes"
ExpiresByType text/javascript "access plus 60 minutes" ExpiresByType application/x-javascript "access plus 60 minutes" ExpiresByType text/xml "access plus 60 minutes"
Initially this block in apache2.conf was inside an if tag, IfModule mod_expires.c - but was not working. Then I commented out the IfModule tags - then it started working, since anyway I had enabled the module using a2enmod expires
Next - wanted to enable some sort of automated log parsing. Among free log analyzers, settled on awstats, since someone had made a mod for wowza too. Followed the ubuntu documentation for awstats installation. There, the recommended cron is
This notation */3 means once every three hours.
For installing GeoIP - tried enabling it directly, since it is supposed to be one of the default included modules, but got this error,
Error: Plugin load for plugin 'geoipfree' failed with return code: Error: Can't locate Geo/IPfree.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . /usr/share/awstats/lib /usr/share/awstats/plugins /usr/lib/cgi-bin/plugins) at (eval 9) line 1.
So now I knew where the plugin directory was - so, could follow the method given here for installing maxmind's geolite databases.
wget http://geolite.maxmind.com/
wget http://geolite.maxmind.com/
wget http://geolite.maxmind.com/
gunzip GeoIP.dat.gz
gunzip GeoLiteCity.dat.gz
gunzip GeoIPASNum.dat.gz
mkdir /usr/local/share/GeoIP/
mv *.dat /usr/local/share/GeoIP/
Tried installing Geo::IP Perl module with
http://www.cpan.org/modules/
So, followed the steps at
http://antezeta.com/awstats/
to install the C module.
wget http://geolite.maxmind.com/
gunzip GeoIP.tar.gz
tar xvf GeoIP.tar.gz
cd GeoIP-1.4.8
./configure
make
make install
Then,
cpanm Geo::IP worked.
Uncommented
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
changed the path to /usr/local/share/GeoIP/GeoIP.
Did not add the other two plugins, for city and ISP, since that would slow down AWstats.
Had to flush newer records, by deleting files in
DataDir - /var/lib/awstats
in order to process older log files, as given at
http://awstats.sourceforge.
Took around 15 minutes to crunch old logs for 15 days, so probably the log for 3 hours will take only half a minute or less.
Next for Wowza. Did not have to modify the logging parameters as given at
http://www.wmconsulting.info/
since most of the parameters were already set. But had to change the LogFormat drastically, since my Wowza version is different - 3.03. Also, one point which he makes is to log only the destroy stream event. Not doing that, logging everything as we are doing, was causing over-counting of bytes sent. So, changed the LogFile parameter to read
LogFile=”/usr/local/awstats-
And my LogFormat parameter is
LogFormat="date time %other %other %other %other %code %other %other %other %virtualname %other %other %other %other %url c-ip %method %referer %ua %other %other sc-bytes %other %other %other %other %other %other %other %other %other %other %other %other %other %other %other"
Using this method, bandwidth is slightly under-reported - by some 10%. But cheaper than Sawmill.
To make pdf reports, on the server, the buildstaticpages tool needed htmldoc, which I installed with apt-get install htmldoc
Initially this block in apache2.conf was inside an if tag, IfModule mod_expires.c - but was not working. Then I commented out the IfModule tags - then it started working, since anyway I had enabled the module using a2enmod expires
Next - wanted to enable some sort of automated log parsing. Among free log analyzers, settled on awstats, since someone had made a mod for wowza too. Followed the ubuntu documentation for awstats installation. There, the recommended cron is
0 */3 * * * /usr/lib/cgi-bin/awstats.pl -config=yourdomain.ext -update > /dev/null
This notation */3 means once every three hours.
For installing GeoIP - tried enabling it directly, since it is supposed to be one of the default included modules, but got this error,
Error: Plugin load for plugin 'geoipfree' failed with return code: Error: Can't locate Geo/IPfree.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . /usr/share/awstats/lib /usr/share/awstats/plugins /usr/lib/cgi-bin/plugins) at (eval 9) line 1.
So now I knew where the plugin directory was - so, could follow the method given here for installing maxmind's geolite databases.
wget http://geolite.maxmind.com/
wget http://geolite.maxmind.com/
wget http://geolite.maxmind.com/
gunzip GeoIP.dat.gz
gunzip GeoLiteCity.dat.gz
gunzip GeoIPASNum.dat.gz
mkdir /usr/local/share/GeoIP/
mv *.dat /usr/local/share/GeoIP/
Tried installing Geo::IP Perl module with
http://www.cpan.org/modules/
cpan App::cpanminus cpanm Geo::IPFAIL Configure failed for Geo-IP-1.40. See /root/.cpanm/build.log for details
So, followed the steps at
http://antezeta.com/awstats/
to install the C module.
wget http://geolite.maxmind.com/
gunzip GeoIP.tar.gz
tar xvf GeoIP.tar.gz
cd GeoIP-1.4.8
./configure
make
make install
Then,
cpanm Geo::IP worked.
Uncommented
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
changed the path to /usr/local/share/GeoIP/GeoIP.
Did not add the other two plugins, for city and ISP, since that would slow down AWstats.
Had to flush newer records, by deleting files in
DataDir - /var/lib/awstats
in order to process older log files, as given at
http://awstats.sourceforge.
Took around 15 minutes to crunch old logs for 15 days, so probably the log for 3 hours will take only half a minute or less.
Next for Wowza. Did not have to modify the logging parameters as given at
http://www.wmconsulting.info/
since most of the parameters were already set. But had to change the LogFormat drastically, since my Wowza version is different - 3.03. Also, one point which he makes is to log only the destroy stream event. Not doing that, logging everything as we are doing, was causing over-counting of bytes sent. So, changed the LogFile parameter to read
LogFile=”/usr/local/awstats-
And my LogFormat parameter is
LogFormat="date time %other %other %other %other %code %other %other %other %virtualname %other %other %other %other %url c-ip %method %referer %ua %other %other sc-bytes %other %other %other %other %other %other %other %other %other %other %other %other %other %other %other"
Using this method, bandwidth is slightly under-reported - by some 10%. But cheaper than Sawmill.
To make pdf reports, on the server, the buildstaticpages tool needed htmldoc, which I installed with apt-get install htmldoc
No comments:
Post a Comment