Log file management and analysis

From WormBaseWiki
Revision as of 18:31, 21 March 2008 by Tharris (talk | contribs) (New page: =Approaches= I've used just about every logfile analysis tool known to man. Here are a few I use regularly. ==Analog== ==awstats== ==Visitors== Website: http://hping.org/visitors Li...)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Approaches

I've used just about every logfile analysis tool known to man. Here are a few I use regularly.

Analog

awstats

Visitors

Website: http://hping.org/visitors

License: Open-source

Installation

wget http://www.hping.org/visitors/visitors-0.7.tar.gz
tar xzf visitors-0.7.tar.gz
cd visitors_0.7
make

Usage

Note: Logs here already contain IP addresses resolved by hostname bruteforce or jdresolve.

todd> zcat /home/todd/projects/wormbase/log_archive/with_hosts/access_log.2008.gz | \
            visitors -A -T --prefix http://www.wormbase.org --prefix http://wormbase.org - \
            > log_analysis.html

To generate a dot representation of trails, try

todd> zcat /home/todd/projects/wormbase/log_archive/with_hosts/access_log.2008.gz | \
            visitors -A --prefix http://www.wormbase.org --prefix http://wormbase.org --graphviz - \
            > log_analysis.html
todd> dot graph.dot -Tpng > graph.png

Mint