Improve Docker recipe for authenticated app (#1518)

DEVELOPMENT.md

@@ -334,3 +334,5 @@ docker push ghcr.io/asreview/asreview:1.0
 ```
 
 If you are creating a Docker container that runs the app with a [config file](#full-configuration) do __not forget__ to override the IP-address of the Flask backend. Set the HOST variable to "0.0.0.0" since the default "localhost" can't be reached from outside the container.
+
+See the `Docker` folder for more information about running the ASReview app in Docker containers.

Docker/README.md

@@ -0,0 +1,69 @@
+# Building ASReview in Docker containers
+
+This folder contains two recipes to build different versions of the ASReview application in a Docker container. The `simple` folder lists a single Dockerfile that builds a simple, non authenticated version of the ASReview app. If you choose to create this container, and multiple people would like to use it, the app will be globally shared amongst all of them. This version makes more sense as a standalone app on your own computer for individual use.
+
+The `auth_verified` folder creates an authenticated version that allows multiple users to access the app and create their own private projects. It requires users to signup and signin in order to access the app.
+
+## Building the simple version
+
+Creating the docker container for a simple, non-authenticated version of the app is done with the following commands (run these commands from the __root__ folder of the app to ensure the correct context):
+
+```
+# create a volume
+$ docker volume create asreview_simple
+# build the container
+$ docker build  -t asreview -f ./Docker/simple/Dockerfile .
+# run container
+$ docker run -d -v asreview_simple_volume:/project_folder -p 8080:5000 asreview
+```
+
+with the external port 8080 being a suggestion. After the last command you find the app in your browser at `http://localhost:8080`.
+
+If you are creating a Docker container that runs the app with a config file, do __not forget__ to override the IP-address of the Flask backend. Set the HOST variable to "0.0.0.0" since the default "localhost" can't be reached from outside the container.
+
+## Building the authenticated, verified version
+
+If you would like to setup the ASReview application as a shared service, a more complicated container setup is required. A common, robust, setup for a Flask/React application is to use [NGINX](https://www.nginx.com/) to serve the frontend, and [Gunicorn](https://gunicorn.org/) to serve the backend. We build separate containers for a database (used for user accounts), and both front- and backend with [docker-compose](https://docs.docker.com/compose/).
+
+For account verification, but also for the forgot-password feature, an email server is required. But maintaining an email server can be demanding. If you would like to avoid it, a third-party service like [SendGrid](https://sendgrid.com/) might be a good alternative. In this recipe we use the SMTP Relay Service from Sendgrid: every email sent by the ASReview application will be relayed by this service. Sendgrid is for free if you don't expect the application to send more than 100 emails per day. Receiving reply emails from end-users is not possible if you use the Relay service, but that might be irrelevant.
+
+In the `auth_verified` folder you find 7 files:
+1. `.env` - An environment variable file for all relevant parameters (ports, database and Gunicorn related parameters)
+2. `asreview.conf` - a configuration files used by NGINX.
+3. `docker-compose.yml` - the docker compose file that will create the Docker containers.
+4. `Dockerfile_backend` - Dockerfile for the backend, installs all Python related software, including Gunicorn, and starts the backend server.
+5. `Dockerfile_frontend` - Dockerfile for the frontend, installs Node, the React frontend and NGINX and starts the NGINX server.
+6. `flask_config.toml` - the configuration file for the ASReview application. Contains the necessary EMAIL_CONFIG parameters to link the application to the Sendgrid Relay Service.
+7. `wsgi.py` - a tiny Python file that serves the backend with Gunicorn.
+
+### SendGrid
+
+If you would like to use or try out [SendGrid](https://sendgrid.com/), go to their website, create an account and sign in. Once signed in, click on "Email API" in the menu and subsequently click on the "Integration Guide" link. Then, choose "SMTP Relay", create an API key and copy the resulting settings (Server, Ports, Username and Password) in your `flask_config.toml` file. It's important to continue with checking the "I've updated my settings" checkbox when it's visible __and__ to click on the "Next: verify Integration" button before you build the Docker containers.
+
+### Parameters in the .env file
+
+The .env file contains all necessary parameters to deploy all containers. All variables that end with the `_PORT` suffix refer to the internal and external network ports of the containers. The prefix of these variable explains for which container they are used. Note that the external port of the frontend container, the container that will be directly used by the end-user, is 8080, and not 80. Change this into 80 if you dont want to use port numbers in the URL of the ASReview web application.
+
+The `EMAIL_PASSWORD` refers to the password provided by the SendGrid Relay service, and the value of the `WORKERS` parameter determines how many instances of the ASReview app Gunicorn will start.
+
+All variables that start with the `POSTGRES` postfix are meant for the PostgreSQL database. The `_USER`, `_PASSWORD` variables are self-explanatory. the `_DB` variable determines the name of the database.
+
+### Creating and running the containers
+
+From the __root__ folder of the app execute the `docker compose` command:
+
+```
+$ docker compose -f ./Docker/auth_verified/docker-compose.yml up --build
+```
+
+### Short explanation of the docker-compose workflow
+
+Building the database container is straightforward, there is no Dockerfile involved. The container spins up a PostgreSQL database, protected by the username and password values in the `.env` file. The backend container depends on the database container to ensure the backend can only start when the database exists.
+
+The frontend container uses a multi-stage Dockerfile. The first phase builds the React frontend and copies it to the second phase which deploys a simple NGINX container. The `asreview.conf` file is used to configure NGINX to serve the frontend.
+
+The backend container is more complicated. It also uses a multi-stage Dockerfile. In the first stage all necessary Python/PostgreSQL related software is installed and the app is build. The app is copied into the second stage. Within the second stage the `flask_config.toml` file is copied into the container and all missing parameters (database-uri and email password) are adjusted according to the values in the `.env` file. The path of this Flask configuration file will be communicated to the Flask app by an environment variable.\
+Then a Gunicorn config file (`gunicorn.conf.py`) is created on the fly which sets the server port and the preferred amount of workers. After that a second file is created: an executable shell script that instructs the ASReview app to create the necessary tables in the database and start the Gunicorn server using the configuration described in the previous file.
+
+Note that a user of this recipe only has to change the necessary values in the `.env` file and execute the `docker compose` command to spin up an ASReview service, without an encrypted HTTP protocol!
+

Docker/auth_verified/.env

@@ -0,0 +1,15 @@
+BACKEND_EXTERNAL_PORT=5015
+BACKEND_INTERNAL_PORT=5005
+
+EMAIL_PASSWORD="password"
+WORKERS=4
+
+FRONTEND_EXTERNAL_PORT=8080
+FRONTEND_INTERNAL_PORT=80
+
+POSTGRES_EXTERNAL_PORT=5433
+POSTGRES_INTERNAL_PORT=5432
+
+POSTGRES_PASSWORD="postgres"
+POSTGRES_USER="postgres"
+POSTGRES_DB="asreview_db"

Docker/auth_verified/Dockerfile_backend

@@ -0,0 +1,70 @@
+# First stage
+FROM python:3.11-slim AS builder
+
+WORKDIR /app
+
+# Copy and build asreview
+# git is used by versioneer to define the project version
+COPY . /app
+RUN rm -rf /app/asreview/webapp/build
+RUN rm -rf /app/asreview/webapp/node_modules
+RUN rm -rf /app/asreview/webapp/public
+RUN rm -rf /app/asreview/webapp/src
+
+RUN apt-get update \
+    && apt-get install -y git build-essential libpq-dev\
+    && pip3 install --upgrade pip setuptools \
+    && pip3 install --user gunicorn \
+    && pip3 install --user . \
+    && pip3 install --user asreview-datatools asreview-insights asreview-makita asreview-wordcloud
+
+
+# Second stage
+FROM python:3.11-slim
+
+# arguments
+ARG EMAIL_PASSWORD
+ARG BACKEND_INTERNAL_PORT_ARG
+ARG WORKERS
+ARG SQLALCHEMY_DATABASE_URI
+ARG CREATE_TABLES
+
+# install necessary libs
+RUN apt-get update && apt-get install -y libpq-dev
+
+WORKDIR /app
+COPY --from=builder /root/.local /root/.local
+
+# copy config TOML file to Image
+COPY ./Docker/auth_verified/flask_config.toml ${WORKDIR}
+
+# the TOML file needs to be configured with the database parameters
+# and email password, we use the sed command to search and insert (replace)
+# necessary parameters
+RUN sed -i "s|--SQLALCHEMY_DATABASE_URI--|${SQLALCHEMY_DATABASE_URI}|g" ./flask_config.toml
+RUN sed -i "s|--EMAIL_PASSWORD--|${EMAIL_PASSWORD}|g" ./flask_config.toml
+
+# set env variables, this is how the TOML config is communicated
+# to the app via Gunicorn
+ENV PATH=/root/.local/bin:$PATH
+ENV ASREVIEW_PATH=/app/project_folder
+ENV FLASK_CONFIGFILE=/app/flask_config.toml
+
+# set the working directory to the app
+WORKDIR /root/.local/lib/python3.11/site-packages/asreview/webapp
+# copy the module that allows Gunicorn to run the app
+COPY ./Docker/auth_verified/wsgi.py ${WORKDIR}
+
+# create Gunicorn config file
+RUN echo "preload_app = True" > gunicorn.conf.py
+RUN echo "bind = \"0.0.0.0:${BACKEND_INTERNAL_PORT_ARG}\"" >> gunicorn.conf.py
+RUN echo "workers = ${WORKERS}" >> gunicorn.conf.py
+
+# create start script to ensure creating all necessary tables and
+# runs the Gunicorn command
+RUN echo "#!/bin/bash" > start.sh
+RUN echo "${CREATE_TABLES}" >> start.sh
+RUN echo "gunicorn -c gunicorn.conf.py wsgi:app" >> start.sh
+RUN ["chmod", "+x", "start.sh"]
+
+ENTRYPOINT [ "/root/.local/lib/python3.11/site-packages/asreview/webapp/start.sh" ]

Docker/auth_verified/Dockerfile_frontend

@@ -0,0 +1,26 @@
+# pull official base image
+FROM node:latest AS builder
+# set working directory
+WORKDIR /app
+# add `/app/node_modules/.bin` to $PATH
+ENV PATH /app/node_modules/.bin:$PATH
+# install app dependencies
+COPY ./asreview/webapp/package.json ./
+COPY ./asreview/webapp/package-lock.json ./
+# Silent clean install of npm
+RUN npm ci --silent
+# add app folders
+COPY ./asreview/webapp/src/ ./src/
+COPY ./asreview/webapp/public/ ./public/
+# create an .env file with backend-url in it
+ARG API_URL
+# Build for production
+RUN REACT_APP_API_URL=${API_URL} \
+  npm run build
+
+# second stage: create nginx container with front-end
+# in it
+FROM nginx:alpine
+ARG API_URL
+COPY --from=builder /app/build /usr/share/nginx/html
+COPY ./Docker/auth_verified/asreview.conf /etc/nginx/conf.d/default.conf

Docker/auth_verified/asreview.conf

@@ -0,0 +1,18 @@
+server {
+    listen       80;
+    server_name  localhost;
+
+    root   /usr/share/nginx/html;
+    index index.html;
+    error_page   500 502 503 504  /50x.html;
+
+    location / {
+        try_files $uri $uri/ /index.html;
+        add_header Cache-Control "no-cache";
+    }
+
+    location /static {
+        expires 1y;
+        add_header Cache-Control "public";
+    }
+}

Docker/auth_verified/docker-compose.yml

@@ -0,0 +1,51 @@
+version: '3.9'
+services:
+
+  asreview_database:
+    container_name: asreview_database
+    image: postgres
+    restart: always
+    environment:
+      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
+      - POSTGRES_USER=${POSTGRES_USER}
+      - POSTGRES_DB=${POSTGRES_DB}
+    ports:
+      - "${POSTGRES_EXTERNAL_PORT}:${POSTGRES_INTERNAL_PORT}"
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -d ${POSTGRES_DB} -U ${POSTGRES_PASSWORD}"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    volumes:
+      - auth_verified_database:/var/lib/postgresql/data
+
+  backend:
+    build:
+      context: ../../
+      dockerfile: ./Docker/auth_verified/Dockerfile_backend
+      args:
+        BACKEND_INTERNAL_PORT_ARG: ${BACKEND_INTERNAL_PORT}
+        EMAIL_PASSWORD: ${EMAIL_PASSWORD}
+        WORKERS: ${WORKERS}
+        SQLALCHEMY_DATABASE_URI: "postgresql+psycopg2://${POSTGRES_USER}:${POSTGRES_PASSWORD}@asreview_database:5432/${POSTGRES_DB}"
+        CREATE_TABLES: "asreview auth-tool create-db postgresql -u ${POSTGRES_USER} -p ${POSTGRES_PASSWORD} -n ${POSTGRES_DB} -H asreview_database"
+    ports:
+      - "${BACKEND_EXTERNAL_PORT}:${BACKEND_INTERNAL_PORT}"
+    depends_on:
+      asreview_database:
+        condition: service_healthy
+    volumes:
+      - auth_verified_project_folder:/app/project_folder
+
+  frontend:
+    build:
+      context: ../../
+      dockerfile: ./Docker/auth_verified/Dockerfile_frontend
+      args:
+        API_URL: http://localhost:${BACKEND_EXTERNAL_PORT}/
+    ports:
+      - "${FRONTEND_EXTERNAL_PORT}:${FRONTEND_INTERNAL_PORT}"
+
+volumes:
+  auth_verified_project_folder:
+  auth_verified_database:

Docker/auth_verified/flask_config.toml

@@ -0,0 +1,24 @@
+DEBUG = true
+HOST = "0.0.0.0"
+PORT = 5000
+AUTHENTICATION_ENABLED = true
+ALLOWED_ORIGINS = ["http://localhost:8080", "http://127.0.0.1:8080"]
+SECRET_KEY = "my_very_secret_key"
+SECURITY_PASSWORD_SALT = "abCDefGH"
+SESSION_COOKIE_SECURE = true
+REMEMBER_COOKIE_SECURE = true
+SESSION_COOKIE_SAMESITE = "Lax"
+SQLALCHEMY_TRACK_MODIFICATIONS = true
+ALLOW_ACCOUNT_CREATION = true
+ALLOW_TEAMS = false
+EMAIL_VERIFICATION = false
+SQLALCHEMY_DATABASE_URI = "--SQLALCHEMY_DATABASE_URI--"
+
+[EMAIL_CONFIG]
+SERVER = "smtp.sendgrid.net"
+PORT = 465
+USERNAME = "apikey"
+PASSWORD = "--EMAIL_PASSWORD--"
+USE_TLS = false
+USE_SSL = true
+REPLY_ADDRESS = "casper@compunist.nl"

Docker/auth_verified/wsgi.py

@@ -0,0 +1,3 @@
+from asreview.webapp.start_flask import create_app
+
+app = create_app()

Dockerfile → Docker/simple/Dockerfile

@@ -1,26 +1,29 @@
 # First stage
-FROM python:3.8-slim AS builder
+FROM python:3.11-slim AS builder
 WORKDIR /app
 
 # Copy and build asreview
 # git is used by versioneer to define the project version
 COPY . /app
 RUN apt-get update \
-    && apt-get install -y git npm \
+    && apt-get install -y git npm libpq-dev\
     && pip3 install --upgrade pip setuptools \
     && python3 setup.py compile_assets \
     && pip3 install --user . \
     && pip3 install --user asreview-datatools asreview-insights asreview-makita asreview-wordcloud
 
 # Second stage
-FROM python:3.8-slim
+FROM python:3.11-slim
+
+VOLUME /project_folder
+
 WORKDIR /app
 
 COPY --from=builder /root/.local /root/.local
 
 ENV ASREVIEW_HOST=0.0.0.0
 ENV PATH=/root/.local/bin:$PATH
-ENV ASREVIEW_PATH=/app/project_folder
+ENV ASREVIEW_PATH=/project_folder
 EXPOSE 5000
 
-ENTRYPOINT ["asreview"]
+ENTRYPOINT ["asreview", "lab"]