Sed search between times


To search between 2 times within an apache log.

Run the below

sed -n '/22\/Aug\/2011:15:40:/,/22\/Aug\/2011:16:00:/ p' accesslog.log
Advertisements

Install goaccess


Goaccess is a handy Apache log analyzer.

To install do the below.

yum install ncurses-devel glib2-devel GeoIP*
cd /usr/src
wget http://sourceforge.net/projects/goaccess/files/0.4/goaccess-0.4.tar.gz/download
tar zxvf goaccess-0.4.tar.gz
cd goaccess-0.4
./configure
make; make install

Once installed then run

goaccess -f "/var/log/youraccess.log"

Command to look at a specific date of a log

sed -n '/05\/Dec\/2010/,$ p' access.log | goaccess -s -b

Screenshot of goaccess in action
Go Access screenshot

Check services for startup linux


To check the services that are set to auto start up on a linux server run the below command.

Run the below

chkconfig --list

Result

apache2         0:off   1:off   2:off   3:off   4:off   5:off   6:off
netdump         0:off   1:off   2:off   3:off   4:off   5:off   6:off
cups-config-daemon      0:off   1:off   2:off   3:off   4:off   5:off   6:off
psacct          0:off   1:off   2:off   3:off   4:off   5:off   6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off
openibd         0:off   1:off   2:on    3:on    4:on    5:on    6:off
dc_client       0:off   1:off   2:off   3:off   4:off   5:off   6:off
rawdevices      0:off   1:off   2:off   3:on    4:on    5:on    6:off
irqbalance      0:off   1:off   2:off   3:off   4:off   5:off   6:off
portmap         0:off   1:off   2:off   3:on    4:on    5:on    6:off
autofs          0:off   1:off   2:off   3:off   4:off   5:off   6:off
sendmail        0:off   1:off   2:on    3:on    4:on    5:on    6:off
messagebus      0:off   1:off   2:off   3:off   4:off   5:off   6:off
cups            0:off   1:off   2:off   3:off   4:off   5:off   6:off
winbind         0:off   1:off   2:off   3:off   4:off   5:off   6:off
diskdump        0:off   1:off   2:off   3:off   4:off   5:off   6:off
acpid           0:off   1:off   2:off   3:on    4:on    5:on    6:off
lm_sensors      0:off   1:off   2:on    3:on    4:on    5:on    6:off
vncserver       0:off   1:off   2:off   3:off   4:off   5:off   6:off
dc_server       0:off   1:off   2:off   3:off   4:off   5:off   6:off
sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
mdmonitor       0:off   1:off   2:on    3:on    4:on    5:on    6:off
readahead       0:off   1:off   2:off   3:off   4:off   5:on    6:off
irda            0:off   1:off   2:off   3:off   4:off   5:off   6:off
bluetooth       0:off   1:off   2:off   3:off   4:off   5:off   6:off
NetworkManager  0:off   1:off   2:off   3:off   4:off   5:off   6:off
yum             0:off   1:off   2:off   3:off   4:off   5:off   6:off
netfs           0:off   1:off   2:off   3:on    4:on    5:on    6:off
lvm2-monitor    0:off   1:on    2:on    3:on    4:on    5:on    6:off
nfs             0:off   1:off   2:off   3:off   4:off   5:off   6:off
pcmcia          0:off   1:off   2:off   3:off   4:off   5:off   6:off
arptables_jf    0:off   1:off   2:on    3:on    4:on    5:on    6:off
snmpd           0:off   1:off   2:on    3:on    4:on    5:on    6:off
microcode_ctl   0:off   1:off   2:on    3:on    4:on    5:on    6:off
ntpd            0:off   1:off   2:on    3:on    4:on    5:on    6:off
atd             0:off   1:off   2:off   3:on    4:on    5:on    6:off
gpm             0:off   1:off   2:off   3:off   4:off   5:off   6:off
ypbind          0:off   1:off   2:off   3:off   4:off   5:off   6:off
firstboot       0:off   1:off   2:off   3:on    4:off   5:on    6:off
readahead_early 0:off   1:off   2:off   3:off   4:off   5:on    6:off
canna           0:off   1:off   2:off   3:off   4:off   5:off   6:off
vsftpd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
sysstat         0:off   1:on    2:on    3:on    4:on    5:on    6:off
iptables        0:off   1:off   2:on    3:on    4:on    5:on    6:off
smartd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
vmware-tools    0:off   1:off   2:on    3:on    4:off   5:on    6:off
haldaemon       0:off   1:off   2:off   3:off   4:off   5:off   6:off
snmptrapd       0:off   1:off   2:off   3:off   4:off   5:off   6:off
kudzu           0:off   1:off   2:off   3:off   4:off   5:off   6:off
anacron         0:off   1:off   2:on    3:on    4:on    5:on    6:off
iiim            0:off   1:off   2:off   3:off   4:off   5:off   6:off
nfslock         0:off   1:off   2:off   3:on    4:on    5:on    6:off
mdmpd           0:off   1:off   2:off   3:off   4:off   5:off   6:off
rdisc           0:off   1:off   2:off   3:off   4:off   5:off   6:off
xinetd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
rpcidmapd       0:off   1:off   2:off   3:on    4:on    5:on    6:off
crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off
saslauthd       0:off   1:off   2:off   3:off   4:off   5:off   6:off
jboss           0:off   1:off   2:off   3:off   4:off   5:off   6:off
auditd          0:off   1:off   2:off   3:off   4:off   5:off   6:off
rpcgssd         0:off   1:off   2:off   3:on    4:on    5:on    6:off
xfs             0:off   1:off   2:off   3:off   4:off   5:off   6:off
netplugd        0:off   1:off   2:off   3:off   4:off   5:off   6:off
cpuspeed        0:off   1:off   2:off   3:off   4:off   5:off   6:off
wpa_supplicant  0:off   1:off   2:off   3:off   4:off   5:off   6:off
FreeWnn         0:off   1:off   2:off   3:off   4:off   5:off   6:off
ipmi            0:off   1:off   2:off   3:off   4:off   5:off   6:off
syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off
isdn            0:off   1:off   2:on    3:on    4:on    5:on    6:off
nscd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
rhnsd           0:off   1:off   2:off   3:off   4:off   5:off   6:off

Awk and more awk..


The logs they never end…More logs and more logs and more problems with more logs…Needed to look at some apache access logs today. Handy to be able to use awk to filter the data to make it a bit more visible to whats going on. Using the below we can count the number of occurances for the specified part of the line. In the below example its getting the number of occurances for the IP’s in the access log but you can amend this to see the request headers or status codes depending on their location on the line using awk.

Example

Access Log

132.17.14.252 - - [09/Sep/2011:04:16:41 +0100] "GET /something.html HTTP/1.1" 200 7031 46785 "-" "-" blah.something.net

Awk

zcat apache.access.log.1.gz | awk {'print $1'} | sort | uniq -c | sort -nr | head -10

Result

 485487 175.12.15.200
    557 216.151.121.50
    506 80.141.40.132
    486 218.156.138.239
    475 79.142.40.105
    452 79.142.40.115
    450 79.142.41.118
    444 218.156.138.238
    441 87.134.71.123
    436 218.156.138.211