Docker Nginx Local

Local usage of the app in Docker with Nginx

TLDR

• app/config/settings/docker_local.py is used to define global variables
• differenciate INTERNAL and EXTERNAL URLS and find which containers need an external and/or an internal URL
• you can connect several Docker containers that are not in the same docker-compose (i.e., front/ and api/ containers) by adding them to the same network.
• use only nginx.conf (the config of the NGINX docker container)
• all queries must be done to the NGINX container. It proxies requests to other containers in the front's docker-compose.

---

Current Prod Docker workflow, in short

Setup

We use

• a docker-compose with an nginx service that is configured to do routing between docker components
• an nginx server on our host that exposes your app through HTTPS.

So basically: Nginx -> Docker -> Nginx -> Other containers

Workflow

cd front/docker
bash docker.sh build

This will:

• prompt you for env variables
• generate NGINX, Supervisord and Gunicorn configs from templates
• build all the docker containers
• run the docker containers.

Adaptations for dev

All NGINX configs are useless except for nginx.conf (the NGINX config of your Docker compose). DO NOT update your global NGINX conf.

---

Changes for a Docker dev deploy

Fix Django app

1. Ctrl+F to requests\. to see where HTTP queries are run in the app. Check which URLs are queried:
   • you must use an INTERNAL URL if it is an URL to another Docker container AND is only accessible through localhost
   • you must use an EXTERNAL URL if it is an URL to something outside your container/network.
   • NOTE: you cannot use localhost.
2. Update front/app/config/settings/docker_local.py with valid INTERNAL/EXTERNAL URLs.

Which URLs to use ?

• When a request is done from the Django backend, use an INTERNAL URL
• When a request is done from a client, use the EXTERNAL URL.

Note that your navigator is a client (which is why things are complicated): all URLs to images in the navigator must use EXTERNAL URLs.

Container	URL pattern	Notes
web	EXTERNAL	AFAIK, there are no queries to Django from Django, so no need for an internal URL
cantaloupe	EXTERNAL and INTERNAL	in Django export functions zip_img, zip_images_and_files URLs redirect to Cantaloupe (server-side). All other Cantaloupe URLs are done in the navigator (client side)
aiiinotate	EXTERNAL and INTERNAL	in Django, all interactions with aiiinotate must be rewritten to INTERNAL. in the navigator, MIRADOR fetches data from aiiinotate and must use an EXTERNAL URL.
mirador	EXTERNAL	Mirador is only ever used client-side, in the navigator
mongo	INTERNAL	mongo is only interacted through aiiinotate, and both are contenerized
API	¯_(ツ)_/¯

Communicate with the API

Currently, the API needs to be exposed to the Web and you cannot use an API that is only accessible to localhost: it is outside of your Docker compose and of your Docker network.

To make the API accessible in localhost, it needs to use the same Docker network as your front's docker-compose. These are the possible options:

1. Add the API to the front's docker-compose
2. Create a shared network between front and API. NOTE: this would need to be run everytime you start the docker.
   1. Create the network:

   docker network create shared-dev-network

   2. Add the API to the network:

   docker network connect shared-dev-network docker-api-1

   3. Update your front's docker-compose:

   networks:
       shared-dev-network:
           external: true

   4. Update your front's NGINX config and settings/docker_local.py if needed.
   5. Maybe update the API's NGINX config as well.

Internal/External URLs

	What	When	Format	Example
INTERNAL	Uses the Docker DNS	Query from a Docker container to another	http://$container_name:$container_port	http://django:8000
EXTERNAL	Doesn't use the Docker DNS	Query to a Docker container from outside the Docker (from a client, or from the host server)	http(s)?://ip:port or http(s?)://domain_name	http://localhost:8080

Explanation:

• querying to localhost from a Docker container refers to the localhost of the Docker container, not to the host machine. So if you are in a Docker container and want to hit the NGINX Docker container running on localhost:8080 on your host, you need to query it using the Docker DNS: nginx:8080. Docker will resolve the container name to what it points to on the machine.
• however, querying to $docker-name outside of a Docker container will not work. If you need to serve images in a cantaloupe container,
  • hitting cantaloupe/path/to/image.jpg will only work from within the Docker container.
  • to query it from outside (i.e., yourWeb Browser), you need to use localhost and go through the NGINX docker container: localhost:8080/cantaloupe/path/to/image.jpg
• this problem doesn't appear in production, when the app is accessible through the Web: in that case, you have a domain name (aikon.enpc.fr), and can do all queries to aikon.enpc.fr. Since your domain name is publicly accesible, a query will look like: Query from Docker container -> aikon.enpc.fr/aiiinotate -> Query arrives on your HTTP server -> NGINX redirects to the aiiinotate container

Examples: running a query to another container named aiiinotate (port: 5555) from a web container (port: 8000).

• CASE 1: using an EXTERNAL URL from a Docker container to another Docker container with localhost

  curl localhost:5555
   ↓
  looks for the localhost of your django container
   ↓
  request fails
  

• CASE 2: using an INTERNAL URL from a Docker container to another Docker container

  curl aiiinotate:5555
   ↓
  django DNS resolves `aiiinotate` to a Docker container and proxies the query to that container
   ↓
  request succeeds ! `localhost:5555` on your host server is queried.
  

• CASE 3: query from one Docker container, in prod, when your app is exposed to the internet:

  curl aikon.enpc.fr/aiiinotate
   ↓
  request arrives to your NGINX docker
   ↓
  the NGINX docker proxies `/aiiinotate/` to the `aiiinotate` docker container
   ↓
  request succeeds  ! 
  

---

---

---

OLD DOCS

In general, Docker and Nginx are used for prod builds.

Here is how to use Docker and Nginx to build the app and access it with http://localhost:8080.

This assumes you are familiar with Docker deploy and have all the necessary packages installed on your server.

---

Prod Docker deploy, in short

Setup

We use

• a docker-compose with an nginx service that is configured to do routing between docker components
• an nginx server on our host that exposes your app through HTTPS.

So basically: Nginx -> Docker -> Nginx -> Other containers

Workflow

cd front/docker
bash docker.sh build

This will:

• prompt you for env variables
• generate NGINX, Supervisord and Gunicorn configs from templates
• build all the docker containers
• run the docker containers.

Changes for localhost usage

In short,

• the host nginx server is useless
• the django settings must be updated to work locally
• all HOSTS in your .env files must have the names of containers, not URIs.
• add optional SSH port forwarding
• PROD_URL in your .env must be localhost.

---

Run docker locally

1. .env variables

Note: only relevant variables are included. In short, set DOCKER=True, PROD_URL=localhost and all *_HOST variables to their docker-compose container name.

# front/app/config/.env

TARGET=prod
DOCKER=True
C_FORCE_ROOT=True
MEDIA_DIR=/data/mediafiles
PROD_URL=localhost  # !! this is important
REDIS_HOST=redis
MONGODB_HOST=redis
AIIINOTATE_HOST=aiiinotate
MIRADOR_HOST=mirador

# font/docker/.env

REDIS_HOST=redis

2. Update your Django settings (front/app/config/settings/base.py)

Comment this block:

ALLOWED_HOSTS = hosts + https_hosts + wildcard_hosts
CSRF_TRUSTED_ORIGINS = https_hosts + wildcard_hosts
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")

Add this block:

USE_X_FORWARDED_HOST = True
CSRF_COOKIE_SECURE = False
SESSION_COOKIE_SECURE = False
ALLOWED_HOSTS = ["localhost", "127.0.0.1", "web"]
CSRF_TRUSTED_ORIGINS = [ "http://localhost:8080" ]

3. Build your containers

In theory, you don't need to change the build pipeline.

cd front/docker
bash docker.sh build

3. Nginx

Several nginx templates are generated. The two main are:

• nginx.conf: internal nginx for the Docker compose
• nginx_external.conf: nginx conf for the host server

a. Don't update your nginx host server !

nginx_external.conf is useless: we want to access the site on localhost, while nginx_external exposes the host server to HTTPs.

b. Ensure your nginx.conf is valid

I.E., that

• server/server_name is localhost
• server/location uses the Docker container names, so that docker can properly redirect requests to containers.

4. (Optional) port forwarding

If you are working on a remote server through SSH and need to access the server's localhost on your machine, you need to do port forwarding.

There are 2 options:

• through a plugin on your IDE, like Open Remote SSH
• through SSH tunnelling.

The SSH tunnelling syntax is:

ssh -L [LOCAL_IP:]LOCAL_PORT:DESTINATION:DESTINATION_PORT [USER@]SSH_SERVER

For example, the command below binds port 8080 of my-super-server to your machine's localhost:8080 and connects to my-super-server as admin.

ssh -L 8080:my-super-server:8080 admin@my-super-server

5. Done !!

Connect to http://localhost:$NGINX_PORT (default nginx port is 8080).