GeoIP log analysis with bash.
Running websites or other online services usually we want to get as much statistics about visitors as possible. There are plenty of system that can provide us web-based statistics. But there is another way. True admin/devops way is to analyse logs.
Any service can provide access logs: nginx, apache, haproxy, etc. We also can have any statistic by logging access via firewall. As the result there are always IPs of the visitors.
The problem: to get GeoIP statistics from nginx access-log in as quick as possible.
Solution in 3 simple steps:
1st. Tools
There are IP lookup command line tools that use the GeoIP library.
Debian/Ubuntu:
1 2 |
$ sudo apt-get update $ sudo apt-get install geoip-bin |
FreeBSD, pkg
1 2 |
$ sudo pkg update $ sudo pkg install geoip |
2nd. GeoIP Database
MaxMind provides free GeoIP Country database. Here we can download the geolite db.
1 2 |
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz $ gunzip GeoIP.dat.gz |
3rd. Analysis
To check single IP:
1 2 |
$ geoiplookup -f GeoIP.dat 8.8.8.8 GeoIP Country Edition: US, United States |
Parsing access-log:
1 |
$ awk '{print $1}' admins247.com-access.log | sort -n | uniq | xargs geoiplookup -f /usr/local/etc/nginx/GeoIP.dat |
This give you a line per country for each IP address.