Real-time Pedestrian Cell Phone Usage Detection - Storage Backend

Alias: YOU FOCUS YOUR WALK

Note

You don't need to manually build the storage backend if you just want to experience the project, because the storage backend is already deployed in the remote Ubuntu server in DigitalOcean.

This README is a manual to deploy the storage backend on an Ubuntu Server droplet in DigitalOcean.

The frontend is run locally. It does not run on this server.

Personnel

• Group Members: Mai Jiajun DC127853, Huang Yanzhen DC126732
• Supervisor: Prof. Bob Zhang

Basic Information

License

This repository uses open-source software licensed under different licenses. The full texts are available in the LICENSES directory.

Software	Type	LICENSE File (Local)	Source (with License)
python-dotenv	BSD 3-Clause	LICENSE	Link
uvicorn	BSD 3-Clause	LICENSE	Link
dlib	BSL-1.0	LICENSE	Link
FastAPI	MIT	LICENSE	Link
opencv-python	MIT	LICENSE	Link
PyJWT	MIT	LICENSE	Link
python-magic	MIT	LICENSE	Link
SQLAlchemy	MIT	LICENSE, AUTHORS	Link
postgres	-	COPYRIGHT	Link

Deployment Info

The storage backend is deployed in the remote server.

• Reverse proxied with Nginx
• Configured SSL certificate with Certbot (quiet auto-renewed every day)
• Deployed on Ubuntu, from infrastructure platform DigitalOcean
• Server Domain: youfocusyourwalk.top

[!NOTICE]

Important notice: If you want to perform interface testing, please use curl to write http request, and don't use the swagger UI of FastApi, as it doesn't support adding authorization to HTTP header.

A good curl tool is the Rest Client extension of VSCode.

Pre-requisites

Pre-1 What to prepare

If you want to deploy the storage backend to your own server, you need to prepare the following yourself.

• A remote Ubuntu server and a domain.
  • You may buy a domain on Porkbun as I do.
  • You need to configure your domain's DNS record to your server's IP. You may refer to the tutorial from DigitalOcean.
• A SECRET_KEY in the environment .env to generate JWT.
  • A secret key is generated by this piece of python code. Please generate it yourself.

import secrets
def generate_secret_key(byte_num: int = 32):    # Don't change the byte_num
    secret_key = secrets.token_urlsafe(byte_num)
    return secret_key

generate_secret_key()

Pre-2 Multiplexing with Screen

Supposed you have already connected to your remote server with SSH in command line. Screen is a software to help you multitask (i.e., multiplex) in a command-line UI.

0. Install Screen.

sudo apt install screen

1. Create a Screen session.

screen -S <SCREEN_NAME>

Then, a terminal session is started. (Like a windows on GUI.)

To detach a screen session, press CTRL + A first, and pres d. Then, the current session is detached. (Like minimizing a window in GUI.)

2. Re-attach screen session.

screen -r <SCREEN_NAME>

Then, you go back to a screen session. (Like re-opening a window in GUI.)

3. See all opened screen session.

screen -ls

You will see all your screen session listed. (Like a task manager in Windows.)

If you list all screen sessions and find something like this:

There are screens on:
        146612.backend-python-code      (03/20/25 08:38:44)     (Detached)
        127124.nginx                    (03/19/25 06:22:13)     (Detached)
        12469.postgreSQL                (03/13/25 12:59:26)     (Detached)
        12222.backend-python            (03/13/25 12:28:22)     (Attached)
4 Sockets in /run/screen/S-root.

There's an Attached session like given above, you can manually detach it like this:

screen -d backend-python

Pre-3 Install Key Software with apt

3.1 Install build tools: build-essential and cmake.

sudo apt-get install build-essential && apt-get install cmake

They are installed to compile opencv and dlib.

3.2 Install wget:

sudo apt install wget

It is a network utility used to download anaconda.

3.3 Install git

sudo apt install git

Configure git.

git config --global user.namd "<YOUR_GITHUB_USERNAME>"
git config --global user.email "<YOUR_GITHUB_EMAIL>"

Logging in to a GitHub account is NOT mandatory to clone this repository, since it's public:

https://github.com/FYP-hyz-mjj-2024/posture-face-compare.git

However, if you prefer, you can use SSH or HTTPS (with personal access token only) to log in to GitHub.

If you prefer to log in to GitHub, I suggest to install a software gh for your convenience.

sudo apt install gh

You can refer to GitHub's tutorial to use SSH or HTTPS to log in your GitHub Account:

Deployment

Dep-1 Install PostgreSQL

Please create/re-attach to a dedicated screen session with an intuitive name. e.g.: screen -r PostgreSQL

1.1 Install postgresql with apt.

sudo apt install postgresql

You can start the postgreSQL process by:

sudo systemctl start postgresql.service

1.2 Create superuser.

sudo -u <USER_NAME> psql template1

ALTER USER <USER_NAME> with encrpted password '<PASSWORD>';

1.3 Restart PostgreSQL

sudo systemctl restart postgresql.service

Similarly, you can replace the restart tag with start or stop.

See more PostgreSQL tips in here. You can also refer to the official documentation.

Please detach this screen session when finished.

Dep-2 Runtime Environment

Please create/re-attach to a dedicated screen session with an intuitive name. e.g.:

screen -r backend-python-code

2.1 Install Conda

2.2.1 Use wget to download the installer. The installer file will be stored in the root file.

wget https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh

2.2.2 Run the installer to install conda.

bash <conda-installer-name>-latest-Linux-x86_64.sh

After installation, restart your ssh console.

2.3 Create Virtual Environment

2.3.1 Create virtual environment.

conda create --prefix <PATH_TO_YOUR_VENV>/<VENV_NAME> python=3.12

2.3.2 Clone this repository:

git clone https://github.com/FYP-hyz-mjj-2024/posture-face-compare.git

2.3.3 Setup environment variables. Add an environment variable file called .env in the root directory of the cloned repository.

nano .env

nano is a text editor that creates a file if it doesn't exist. You will see a text editing interface after you typed the above command.

Paste (press Shift+Insert on your keyboard) and fill in this structure in the file:

# Secret key
SECRET_KEY=XXXX             # to generate jwt

# Server Domain
SERVER_DOMAIN=https://www.<YOUR_DOMAIN>
SERVER_HOST=0.0.0.0         # Expose to internet
DATABASE_URL=postgresql://<USER_NAME>:<PASSWORD>@localhost:5432/<DATABASE_NAME>

If you are confused by the https text in the 5th line, don't worry, feel free to write it. We will configure HTTPS later.

2.3.4 Install packages listed in requirements.txt

<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/pip install -r requirements.txt

This process should be done after the 1. Install PostgreSQL and 2. cmake and build-essential, as a python package will always require PostgreSQL and some building tools.

2.3.5 Install additional packages not listed in requirements.

<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/pip install magic
<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/pip install opencv-python-headerless

Please detach this screen session when finished.

Dep-3 Security

Please create/re-attach to a dedicated screen session with an intuitive name. e.g.: screen -r nginx

3.1 Nginx

3.1.1 Install Nginx.

Nginx is a reverse-proxy tool to enhance the safety of the backend.

sudo apt install nginx

3.1.2 Add Nginx configuration.

First, go to the sites-available file of nginx.

cd /etc/nginx/sites-available

Delete the default configuration.

sudo rm ./<default_config_name>

Initialize a new configuration.

nano /etc/nginx/sites-available/<config_name>

nano is a text editor, it will open a blank file if a file doesn't exist. Here, you are actually initializing the configuration file.

3.1.3 Write configuration.

Paste (press Shift + Insert) the following into the configuration file.

server {
    listen 80;
    server_name www.<YOUR_DOMAIN> <YOUR_DOMAIN>;

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }
}

After writing this, close the editor by pressing Ctrl +x (means close) and then y (confirm write).

3.1.4 Apply configuration.

Run the following command to apply your changes in Nginx.

sudo ln /etc/nginx/sites-available/<config_name>
sudo nginx -t
sudo systemctl start nginx

1. First command: Link the config file to Nginx.
2. Second command: Check syntax errors in config file. (Please always do this.)
3. Third command: Runs Nginx to apply changes.

You can also check Nginx's status by running:

sudo systemctl status nginx

Whenever a change in the configuration is done, please reload Nginx.

sudo nginx -t
sudo systemctl reload nginx

3.2 HTTPS

3.2.1 Install Certbot + Plugin

sudo apt install certbot python3-certbot-nginx

Certbot helps you get a static TLS certificate to support HTTPS.

The plugin integrates certbot with Nginx to make configuration more convenient.

Please make sure your domain (in my case, youfocusyourwalk.top), is DNS-propagated correctly, and Nginx is serving HTTP. You may:

1. Ping your domain first by running ping <YOUR_DOMAIN>
2. Test in the browser (or use curl if you prefer) on http://<YOUR_DOMAIN> .

3.2.2 Run Certbot

sudo certbot --nginx -d youfocusyourwalk.top -d www.youfocusyourwalk.top

It will ask you for a few things:

• Your email: Notify you when certificate expires.
• Terms of service: Just agree by typing A
• Whether to choose HTTPS: Please choose HTTPS only, instead of both HTTP and HTTPS.

3.2.3 Auto Renewal

I configured auto-renewal of SSL certificate in the cloud deployment. It is recommended that you do the same thing.

First, test the renewal by running:

sudo certbot renew --dry-run

--dry-run means Certbot will only test the ability to renew the certificate, instead of actually renewing it. Again, please make sure your domain is DNS-propagated for a decent amount of time before you do this.

If the dry run succeeds, edit the root crontab of the system:

sudo crontab -e

Crontab is a system daemon to run specific command at a specific time. Press the link above for an official tutorial.

Add this line to the bottom:

0 3 * * * certbot renew --quiet --post-hook "systemctl reload nginx"

This registers a command to the system that, every day at 3 a.m. (UTC+0), the certificate is renewed.

• --quiet: No outputs unless there's an error.
• --post-hook: Defines the what to do after running this command. (i.e., in this case, reloads Nginx)
• Please notice that DigitalOcean uses UTC+0. So if you uses DigitalOcean and you are in East Asia, you may need to add 8 (Beijing Time) or 9 (Tokyo Time) in hours when you define your work.

Verify crontab with:

sudo crontab -l

This will list all your registered jobs. See if the renewal task is added successfully.

Please detach this screen session when finished.

Dep-4 Run & Maintain

4.1 Setup Database

4.1.1 (In the database screen) Setup database.

sudo -u <USER_NAME> psql

This will enter the edit mode of PostgreSQL. To exit, type \q.

CREATE DATABASE <DATABASE_NAME>;
\c <DATABASE_NAME>

\c means to enter the context of the given database (i.e., use database <DATABASE_NAME>; in MySQL).

Make database support uuid (after \c to enter edit mode in this database)

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

4.1.2 (In the python backend screen) Initialize tables.

Make sure you are in the root directory of the storage backend.

cd <PROJECT_ROOT>

Initialize all tables using python.

<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/python -m CRUD.user.__reset__
<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/python -m CRUD.face.__reset__

4.2 Run Project

<PATH_TO_YOUR_VENV>/<VENV_NAME>/bin/python -m main

Then, detach the current Screen session to maintain running by pressing CTRL+A then d.

Dep-5 PostgreSQL Command Line Tips

5.1 Starting and stopping (For local)

stop database (local only, depends on where your PostgreSQL is installed)

pg_ctl -D "D:\ProgramFiles\PostgreSQL\17\data" stop

start database (local only, depends on where your PostgreSQL is installed)

pg_ctl -D "D:\ProgramFiles\PostgreSQL\17\data" start

5.2 Common Usages

5.2.1 Login as super user postgres

psql -U <USER_NAME>

5.2.2 Common navigations.

PostgreSQL	MySQL Equivalent	Meaning
\l	SHOW DATABASES	List all the databases.
\c <DATABASE_NAME>	USE <DATABASE_NAME>	Enter a database's context
\dt	SHOW TABLES	List all tables in the current database.

5.2.3 Exit database edit mode

\q

5.2.4 Update table info

UPDATE <TABLE_NAME> SET <COLUMN_NAME>=<VALUE> WHERE <CONDITION>