Production-Deployment-ML-Model

A tutorial on deploying any ML model with Flask + Gunicorn + Nginx.

Folder structure

The ML model we have

Given a text corpus, we run word2vec model on it. After tuning our parameters, we save our word2vec model as - model.save("models/w2v.model") Now in any environment, all we need to do is -

    model = w2v.load('models/w2v_bi_v10.model')
    word = "doctor"
    prediction = model.wv.most_similar(word)

The prediction will contain list of top words with similarity score that are similar to word "doctor".

Goal

We need to deploy a simple UI which intakes a word from user and then provides list of words similar to that word, based on our trained word2vec model.

Getting Started

Creating conda environment

conda create -n myapp
conda activate myapp
conda install flask
conda install gensim 

# install lib as per requirements

• As we are using gensim, make sure the version is >4.0 with latest python(3.10)
• To update python packages -
  sudo apt install python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools
• To upgrade from a older gensim version in env -

conda remove --force gensim
pip install  --upgrade gensim --pre

Gunicorn installation

Run cmd

pip install gunicorn
Note make sure to run this command while inside env as it can cause compatibility issues.

Creating our application - app.py

Flask API will receive user input in form of request.form["word"] through UI or API calls, computes the similar words to input based on our saved model and returns it. In app.run(debug=True,host='0.0.0.0') we can change the host as well as port number at which it gets deployed. Follow this cheat-sheet for flask commands.

python app.py

Now our application running successfully on machine_ip:port(by default 5000), go to url http://localhost:5000.

Note: templates/index.html - This folder contains the HTML template to allow user to enter any word and displays the predictions.
Flask server is a development server i.e meant to test locally only. We need to create client-server framework to handle user requests. For this, we need a WSGI and a web server i.e we have -

• Flask:python framework for web application
• Gunicorn : Application server to handle client request
• Nginx: Web server

Connecting app with Gunicorn

So far, we have completed our local deployment with Flask. Now will connect/map our flask application and gunicorn server via connector.py.

gunicorn --bind :5000 connector:app 

This will run gunicorn server by providing connector file and application module name. Application still runs on same url but now the client request gets handled by the app server - Gunicorn.

Before running nginx, we need to setup gunincorn such that it can be started by systemd. This is requirement of nginx.

• Edit service file - sudo vi /etc/systemd/system/myapp.service
  • Working directory - the root folder path
  • Environment - conda env path, can be found via command conda env list
  • ExecStart - <path of gunciron in your env> --workers 3 --bind unix:myapp.sock -m 007 connector:app
    Notice how this gunicorn cmd is different from earlier. Now, we bind it to a unix socket i.e myapp.sock
• To start the gunicorn service, we run following commands -

          sudo systemctl start myapp
          sudo systemctl enable myapp
          sudo systemctl status myapp
  

• The above cmds creates myapp.sock in your my_code folder.
  NOTE: change the permission of this file to 777 so that its accessible to nginx. You might need to change the folder permission access as well, because we see in nginx setuo that its start from root folder("/") and then moves to our unix socket "myapp.sock", we need to make sure nginx has permission to do so.

Setting up our web server - nginx

Steps to follow -

• To install nginx, follow the steps mentioned in Installing nginx on Ubuntu. You can install it outside environment as well.
• sudo usermod ubuntu -g www-data
  • the user here is ubuntu and group is www-data. Change accordingly.
• Create nginx configuration file - sudo vi /etc/nginx/sites-available/myapp
• Create a symlink between /etc/nginx/sites-available and /etc/nginx/sites-enabled
  sudo ln -s /etc/nginx/sites-available/myapp /etc/nginx/sites-enabled
  To check if the linkage is successful, run ls-ln. If there is already symlink and you need to ovewrite then instead of passing -s , pass -sf as argument.
• sudo nginx -t Checks any syntax errors in configuration file: nginx.conf
• Start the application -
  sudo systemctl start nginx
  Application will be running at http://localhost:5000
• A brief explanation of myapp that starts the nginx service

server {
listen 80;
server_name <pass the server ip here> ;  

location / {
       include proxy_params;
       proxy_pass http://unix:/home/deeksha/my_code/myapp.sock;
}
}
 

• listen 80: port at which our app will run
• server_name: machine/server ip adress, therfore our app will run at http:server_ip:80
• proxy_pass: unix socket to transfer request that nginx listen at port 80.
• This file instructs Nginx to listen on port 80 and transfer the request to our api that we created using gunicorn+flask i.e myapp.sock.

Additional functions

Error and process logs

• sudo less /var/log/nginx/error.log: checks the Nginx error logs.
• sudo less /var/log/nginx/access.log: checks the Nginx access logs.
• sudo journalctl -u nginx: checks the Nginx process logs.
• sudo journalctl -u myapp: checks your Flask app’s Gunicorn logs.

References

• Part 1:Deploy Flask App+Anaconda+Gunicorn+Nginx on Ubuntu
• Part 2
• For Flask deployment and UI - flask - deployment