Visualize Nginx web Log with GoAccess

Install GoAccess (Ubuntu 18.04)

Using apt

$ sudo apt install goaccess

For the latest version:

$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
$ wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install goaccess

From source

Install the dependencies:

$ sudo apt-get install libgeoip-dev libncursesw5-dev

Download, configure and install goaccess:

$ wget https://tar.goaccess.io/goaccess-1.4.tar.gz
$ tar -xzvf goaccess-1.4.tar.gz
$ cd goaccess-1.4/
$ ./configure --enable-utf8 --enable-geoip=legacy
$ make
$ sudo make install

Determine Log Format

Once you have GoAccess installed, you should be ready to start using it. However, first you need to determine the log format of your access log. GoAccess comes with several predefined log format options that you can use. Either you can set them permanently in your configuration file or simply passing it through the command line.

Custom log format

goaccess config for custom log format: https://goaccess.io/man#custom-log

Run GoAccess

Terminal Output

The following prompts a log configuration dialog with predefined log formats for you to choose from and then displays the stats in real-time.

goaccess access.log -c

Static HTML Output

The following parses the access log and displays the stats in a static HTML report.

goaccess access.log -o report.html --log-format=COMBINED

Real-Time HTML Output

The following parses the access log and displays the stats in a real-time HTML report.

goaccess access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-html

From nginx log format to goaccess

Small shell script to convert Nginx log_format to goaccess config can be found here:

Usage:

$ ./nginx2goaccess.sh '<log_format>'

For example:

$ ./nginx2goaccess.sh '"$time_local" client=$remote_addr method=$request_method request="$request" request_length=$request_length status=$status bytes_sent=$bytes_sent body_bytes_sent=$body_bytes_sent referer=$http_referer user_agent="$http_user_agent" upstream_addr=$upstream_addr upstream_status=$upstream_status request_time=$request_time upstream_response_time=$upstream_response_time upstream_connect_time=$upstream_connect_time upstream_header_time=$upstream_header_time'

The output will be something like this:

time-format %T
date-format %d/%b/%Y
log_format "%d:%t %^" client=%h method=%m request="%r" request_length=%r_length status=%s bytes_sent=%b body_bytes_sent=%b referer=%R user_agent="%u" upstream_addr=%^ upstream_status=%^ request_time=%T upstream_response_time=%^ upstream_connect_time=%^ upstream_header_time=%^

References

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Expert session

Connect your Particle Photon Directly to Azure IoT Hub or IoT Central

How to Lazy Load Images with an Edge Worker

4 Free CDN Hosting Providers to Speed Up Your Business Website

Deep Learning on ARM Processors

What You Should Know about the Python List

A responsive email workflow in less than 5 minutes!

Simplifying Microservices with Istio in Google Kubernetes Engine — Part III

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Chi Thuc Nguyen

Chi Thuc Nguyen

More from Medium

Authorization vs Authentication

Building a CI-CD Pipeline with Vercel and GitHub Actions

Creating Nested Queries in PostgreSQL

Improving Wireline Service Supply Chain Workflows with Automation