Sed search between times


To search between 2 times within an apache log.

Run the below

sed -n '/22\/Aug\/2011:15:40:/,/22\/Aug\/2011:16:00:/ p' accesslog.log
Advertisements

Install goaccess


Goaccess is a handy Apache log analyzer.

To install do the below.

yum install ncurses-devel glib2-devel GeoIP*
cd /usr/src
wget http://sourceforge.net/projects/goaccess/files/0.4/goaccess-0.4.tar.gz/download
tar zxvf goaccess-0.4.tar.gz
cd goaccess-0.4
./configure
make; make install

Once installed then run

goaccess -f "/var/log/youraccess.log"

Command to look at a specific date of a log

sed -n '/05\/Dec\/2010/,$ p' access.log | goaccess -s -b

Screenshot of goaccess in action
Go Access screenshot

Awk and more awk..


The logs they never end…More logs and more logs and more problems with more logs…Needed to look at some apache access logs today. Handy to be able to use awk to filter the data to make it a bit more visible to whats going on. Using the below we can count the number of occurances for the specified part of the line. In the below example its getting the number of occurances for the IP’s in the access log but you can amend this to see the request headers or status codes depending on their location on the line using awk.

Example

Access Log

132.17.14.252 - - [09/Sep/2011:04:16:41 +0100] "GET /something.html HTTP/1.1" 200 7031 46785 "-" "-" blah.something.net

Awk

zcat apache.access.log.1.gz | awk {'print $1'} | sort | uniq -c | sort -nr | head -10

Result

 485487 175.12.15.200
    557 216.151.121.50
    506 80.141.40.132
    486 218.156.138.239
    475 79.142.40.105
    452 79.142.40.115
    450 79.142.41.118
    444 218.156.138.238
    441 87.134.71.123
    436 218.156.138.211