Visualize Nginx web Log with GoAccess
Install GoAccess (Ubuntu 18.04)
Using apt
$ sudo apt install goaccess
For the latest version:
$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
$ wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install goaccess
From source
Install the dependencies:
$ sudo apt-get install libgeoip-dev libncursesw5-dev
Download, configure and install goaccess:
$ wget https://tar.goaccess.io/goaccess-1.4.tar.gz
$ tar -xzvf goaccess-1.4.tar.gz
$ cd goaccess-1.4/
$ ./configure --enable-utf8 --enable-geoip=legacy
$ make
$ sudo make install
Determine Log Format
Once you have GoAccess installed, you should be ready to start using it. However, first you need to determine the log format of your access log. GoAccess comes with several predefined log format options that you can use. Either you can set them permanently in your configuration file or simply passing it through the command line.
Custom log format
goaccess
config for custom log format: https://goaccess.io/man#custom-log
Run GoAccess
Terminal Output
The following prompts a log configuration dialog with predefined log formats for you to choose from and then displays the stats in real-time.
goaccess access.log -c
Static HTML Output
The following parses the access log and displays the stats in a static HTML report.
goaccess access.log -o report.html --log-format=COMBINED
Real-Time HTML Output
The following parses the access log and displays the stats in a real-time HTML report.
goaccess access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-html
From nginx log format to goaccess
Small shell script to convert Nginx log_format to goaccess config can be found here:
Usage:
$ ./nginx2goaccess.sh '<log_format>'
For example:
$ ./nginx2goaccess.sh '"$time_local" client=$remote_addr method=$request_method request="$request" request_length=$request_length status=$status bytes_sent=$bytes_sent body_bytes_sent=$body_bytes_sent referer=$http_referer user_agent="$http_user_agent" upstream_addr=$upstream_addr upstream_status=$upstream_status request_time=$request_time upstream_response_time=$upstream_response_time upstream_connect_time=$upstream_connect_time upstream_header_time=$upstream_header_time'
The output will be something like this:
time-format %T
date-format %d/%b/%Y
log_format "%d:%t %^" client=%h method=%m request="%r" request_length=%r_length status=%s bytes_sent=%b body_bytes_sent=%b referer=%R user_agent="%u" upstream_addr=%^ upstream_status=%^ request_time=%T upstream_response_time=%^ upstream_connect_time=%^ upstream_header_time=%^
Can be used with -p
and read from a config file or placed into ~/.goaccessrc.