GeoIP log analysis with bash

GeoIP log analysis with bash.

Running websites or other online services usually we want to get as much statistics about visitors as possible. There are plenty of system that can provide us web-based statistics. But there is another way. True admin/devops way is to analyse logs.
Any service can provide access logs: nginx, apache, haproxy, etc. We also can have any statistic by logging access via firewall. As the result there are always IPs of the visitors.

The problem: to get GeoIP statistics from nginx access-log in as quick as possible.
Solution in 3 simple steps:

1st. Tools
There are IP lookup command line tools that use the GeoIP library.

Debian/Ubuntu:

$ sudo apt-get update
$ sudo apt-get install geoip-bin

1 2	$ sudo apt-get update $ sudo apt-get install geoip-bin

FreeBSD, pkg

$ sudo pkg update
$ sudo pkg install geoip

1 2	$ sudo pkg update $ sudo pkg install geoip

2nd. GeoIP Database
MaxMind provides free GeoIP Country database. Here we can download the geolite db.

$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
$ gunzip GeoIP.dat.gz

1 2	$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz $ gunzip GeoIP.dat.gz

3rd. Analysis

To check single IP:

$ geoiplookup -f GeoIP.dat 8.8.8.8
GeoIP Country Edition: US, United States

1 2	$ geoiplookup -f GeoIP.dat 8.8.8.8 GeoIP Country Edition: US, United States

Parsing access-log:

$ awk '{print $1}' admins247.com-access.log  | sort -n | uniq | xargs geoiplookup -f /usr/local/etc/nginx/GeoIP.dat

1	$ awk '{print $1}' admins247.com-access.log \| sort -n \| uniq \| xargs geoiplookup -f /usr/local/etc/nginx/GeoIP.dat

This give you a line per country for each IP address.