Deploy a Python Flask Restful API app with gunicorn, supervisor and nginx

Note: this is a manual procedure to deploy Python Flask app with gunicorn, supervisord and nginx. A full automated CI/CD method is described in another post.

Login to server and clone the source repository

Generate SSH key pair

Login to server and generate new ssh key pair for deployment.

ssh-keygen -t rsa -C "your_email@example.com"

Then locate the private & public keys in ~/.ssh/ folder. You should backup your private key at ~/.ssh/id_rsa.

Added the public key (~/.ssh/id_rsa.pub) as deployment key in your git server. If you are using gitlab you can goto Settings > Repositoty > Deploy keys as following:

Clone the source repository

git clone {project_url} {project_folder}

Setup Python virtual environment and install project dependencies

We use Python 3.7 here.

apt install python3.7
apt install virtualenv
cd {project_folder}
virtualenv -p python3.7 ./.venv
source ./.venv/bin/activate
python --version
pip -V

Or:

apt install python3.7
apt install python3.7-venv
cd {project_folder}
python3.7 -m venv ./.venv
source ./.venv/bin/activate
python --version
pip -V

Install requirements

pip install -r requirements.txt

Setup & configure gunicorn

While lightweight and easy to use, Flask’s built-in server is not suitable for production as it doesn’t scale well. One of the best options available for properly running Flask in production is gunicorn.

Gunicorn ‘Green Unicorn’ is a WSGI HTTP Server for UNIX. It’s a pre-fork worker model ported from Ruby’s Unicorn project. It supports both eventlet and greenlet.

Assume that your app entry point is wsgi.py and there is an application object calledappcreated in this file. If app is missing, gunicorn will get the default value of application.

Running a Flask application on this server is quite simple:

pip install gunicorn
gunicorn -b localhost:8880 -w 4 wsgi:app

dotenv environment variables loading

If you want to load environment variables from .env file using dotenv, you should add this loader script at the top of wsgi.py:

from dotenv import load_dotenv, find_dotenv
load_dotenv(find_dotenv())

from main import app as application

if __name__ == '__main__':
    application.run()

Sample gunicorn config file

# file gunicorn.conf.py
# coding=utf-8
# Reference: https://github.com/benoitc/gunicorn/blob/master/examples/example_config.py
import os
import multiprocessing

_ROOT = os.path.abspath(os.path.join(
    os.path.dirname(__file__), '..'))
_VAR = os.path.join(_ROOT, 'var')
_ETC = os.path.join(_ROOT, 'etc')

loglevel = 'info'
# errorlog = os.path.join(_VAR, 'log/api-error.log')
# accesslog = os.path.join(_VAR, 'log/api-access.log')
errorlog = "-"
accesslog = "-"

# bind = 'unix:%s' % os.path.join(_VAR, 'run/gunicorn.sock')
bind = '0.0.0.0:5000'
# workers = 3
workers = multiprocessing.cpu_count() * 2 + 1

timeout = 3 * 60  # 3 minutes
keepalive = 24 * 60 * 60  # 1 day

capture_output = True

Then start gunicorn app with -c option

gunicorn -c etc/gunicorn.conf.py wsgi

Supervisor

supervisor is a preferable solution to run the gunicorn server in the background and also start it automatically on reboot.

Install supervisor

sudo apt install supervisor

Check if supervisor service is running

service supervisor status

Add a new sudo user

adduser apiuser
adduser apiuser sudo

Change owner of the project folder to newly created user:

chown -R apiuser:apiuser {project_folder}

In this tutorial, project_folder is /opt/deployment/my-api-app.

Create a configuration file for our API project

sudo vim /etc/supervisor/conf.d/my-api-app.conf

… with the following sample content:

;/etc/supervisor/conf.d/my-api-app.conf
[program:my_flask_api_app]
user = apiuser
directory = /opt/deployment/my-api-app
command = /opt/deployment/my-api-app/run.sh gunicorn -c etc/gunicorn.conf.py wsgi

priority = 900
autostart = trueprint(s1)
autorestart = true
stopsignal = TERM

redirect_stderr = true
stdout_logfile = /opt/deployment/my-api-app/var/log/%(program_name)s.log
stderr_logfile = /opt/deployment/my-api-app/var/log/%(program_name)s.log

The run.sh script above is just a simple wrapper to activate Python virtual environment before actually execute the upcoming command. The content of run.sh can be like this:

#!/bin/bash -e

if [ -f .venv/bin/activate ]; then
    echo   "Load Python virtualenv from '.venv/bin/activate'"
    source .venv/bin/activate
fi
exec "$@"

Start gunicorn app with supervisor

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl avail
sudo supervisorctl restart my_flask_api_app

In case you want to reload whole supervisor service:

sudo service supervisor restart

nginx

gunicorn applications stand alone when they run; you can proxy to them from your web server. nginx is one of the best options for this kind of HTTP proxy.

Install nginx

sudo apt install nginx

Double check to make sure the nginx service is running with command service nginx status, then open your browser and enter url http://{server_ip}, you should see some welcome message from nginx.

Configure nginx

sudo touch /etc/nginx/sites-available/api_project
sudo ln -s /etc/nginx/sites-available/api_project /etc/nginx/sites-enabled/api_project
sudo vim /etc/nginx/sites-available/api_project

Sample configuration:

server {
    listen      80;
    server_name api.example.com;
    location / {
        proxy_pass         "http://localhost:5000";
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        fastcgi_read_timeout 300s;
        proxy_read_timeout 300;
    }
    location /static {
        alias  /opt/deployment/my-api-app/static/;
    }
    error_log  /var/log/nginx/api-error.log;
    access_log /var/log/nginx/api-access.log;
}

Check configuration, then restart nginx:

sudo nginx -t
sudo service nginx restart

Now, you should access your api through port 80 with your domain name: http://api.example.com.

Congratulations! Now the Flask app is successfully deployed using a configured nginx, gunicorn and supervisor.

Note: this is a manual procedure to deploy Python Flask app with gunicorn, supervisord and nginx. A full automated CI/CD method is described in another post.

Bonus

Install MySQL

sudo apt install mysql-server

Setup root password

sudo mysql_secure_installation

Note that in Ubuntu systems running MySQL 5.7+, the root MySQL user is set to authenticate using the auth_socket plugin by default rather than with a password. This allows for some greater security and usability in many cases, but it can also complicate things when you need to allow an external program (e.g., phpMyAdmin) to access the user.

If you prefer to use a password when connecting to MySQL as root, you will need to switch its authentication method from auth_socket to mysql_native_password. To do this, open up the MySQL prompt from your terminal:

sudo mysql

Next, check which authentication method each of your MySQL user accounts use with the following command:

mysql> SELECT user,plugin,host FROM mysql.user;

Output:

+------------------+-----------------------+-----------+
| user             | plugin                | host      |
+------------------+-----------------------+-----------+
| root             | auth_socket           | localhost |
| mysql.session    | mysql_native_password | localhost |
| mysql.sys        | mysql_native_password | localhost |
| debian-sys-maint | mysql_native_password | localhost |
+------------------+-----------------------+-----------+
4 rows in set (0.00 sec)

To configure the root account to authenticate with a password, run the following ALTER USER command:

mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'my-password';
mysq> FLUSH PRIVILEGES;

After configuring your root MySQL user to authenticate with a password, you’ll no longer be able to access MySQL with the sudo mysql command used previously. Instead, you must run the following:

mysql -uroot -p

After entering the password you just set, you will see the MySQL prompt.

Install build essentials

sudo apt install build-essential python3.7-dev

Install Redis

sudo apt install redis-server

Test redis

redis-cli

127.0.0.1:6379> ping
PONG
127.0.0.1:6379>

and

redis-cli info
redis-cli info stats
redis-cli info server

Refs

Deployment Options - Flask 1.0.2 documentation

Kickstarting Flask on Ubuntu - Setup and Deployment - Real Python

How To Install and Secure Redis on Ubuntu 18.04 | DigitalOcean