Awk and more awk..


The logs they never end…More logs and more logs and more problems with more logs…Needed to look at some apache access logs today. Handy to be able to use awk to filter the data to make it a bit more visible to whats going on. Using the below we can count the number of occurances for the specified part of the line. In the below example its getting the number of occurances for the IP’s in the access log but you can amend this to see the request headers or status codes depending on their location on the line using awk.

Example

Access Log

132.17.14.252 - - [09/Sep/2011:04:16:41 +0100] "GET /something.html HTTP/1.1" 200 7031 46785 "-" "-" blah.something.net

Awk

zcat apache.access.log.1.gz | awk {'print $1'} | sort | uniq -c | sort -nr | head -10

Result

 485487 175.12.15.200
    557 216.151.121.50
    506 80.141.40.132
    486 218.156.138.239
    475 79.142.40.105
    452 79.142.40.115
    450 79.142.41.118
    444 218.156.138.238
    441 87.134.71.123
    436 218.156.138.211