Skip to content

Demo Server Guide

Jump to bottom
Carl Smith edited this page Nov 6, 2023 · 21 revisions

Conforma Server build, versioning, and install using Docker

The following is a guide to building Conforma, pushing it to the servers, configuring the servers with Docker and docker-compose, and launching the app.

Most of this can be accomplished using custom scripts we've made which reduces the effort required to upgrade a server to just a few short lines. Please see Server Upgrade Guide for an overview of that process, as well as how to configure a new server with the launch scripts.

Prep

  • Add token to githubtoken.txt (for downloading expression-evaluator)
    • requires token that has read access to packages on server repo
    • Troubleshooting error 403 Unauthorised when running yarn install Source:
      • Run npm login --scope=@msupply-foundation --registry=https://npm.pkg.github.com
      • Then enter your github registration username, passowrd (token) and email
  • Ensure your back-end .env has a variable called FRONT_END_PATH with the full path to the front-end repo on your local system.
    e.g. FRONT_END_PATH='/Users/<username>/GitHub/conforma/conforma-web-app/'
  • Ensure your local Docker installation is logged in with the msupplyfoundation account (Password in Bitwarden)
  • Ensure your docker Engine running locally to build a new image
  • Yarn should be in the latest v1. i.e: 1.22.18 to correctly use yarn release

Bump the version and generate a tag.

The version field in package.json is the source of truth for the current version number. Increases to version numbers should be done according to semantic versioning, including "pre-release" versions (for testing purposes, for example).

  • Bump the version and create a tag by running the script yarn release <version-type>, where version-type is on of "--major", "--minor", "--patch", "--prerelease", "--preminor", "--premajor", "--prepatch". ("--pre-release" is the default if not specified.) (See https://classic.yarnpkg.com/en/docs/cli/version for version type explanation)
    • Example: yarn release from develop to generate a Pre-release (release candidate) Or yarn release --minor from main to generate a Minor release.
  • This will also apply the new version number to the front-end (on whatever branch is currently checked out, so it should normally be develop or main). The tag is auto-generated by the yarn version command (run as part of this release script), and is just v<version-num> (e.g. v0.1.5-2)
  • Note: it is not necessary to create a "release" on Github for each tag unless we need to make a particular version publicly downloadable, but one can easily be created based on the current tag/version on Github if required.
  • If anything goes wrong during the release process, you can revert with the following list of commands:
    • git reset --hard HEAD~1 (to revert release bump)
    • git tag -d <tag-name(s)> (delete local tags)
    • git push --delete origin <tag-name(s)> (delete remote tags)
    • You will also need to do the same on the front-end repo.

Build (and upload to Docker hub)

yarn dockerise <tag-or-branch-name> [push]

Example:
yarn dockerise v0.1.5 push

(If you don't specify any parameters, it will build the current develop branch.)

Note that the previously-mentioned yarn release script will offer to start this build process automatically with the tag you've just created.

If building from a branch (as opposed to a tag), be aware of the following:

  • the branch name cannot have a # character in it, like our branches normally begin with.
  • the front-end and back-end branch need to have the same name, so if you're not building from develop or main, you'll probably need to create a "dummy" branch on the other repo so the branch names match.

Login to docker

It should take a while to run and will build a local Docker image. The optional parameter push specifies if the script should automatically push the created image to the msupplyfoundation Docker Hub account. If you are logged to a different acount you need to docker logout and then docker login to your local Docker installation with the msupplyfoundation account (Password in Bitwarden).

Docker image tag

The build process will create a local image with a tag of the form:
build-<releaseTag>_<date>_<uniqueId>

(where uniqueId is a random 6-char string)

To see what's going on under the hood when this command is run, please inspect the file /docker/dockerise.sh

Manually pushing to dockerhub

If you don't auto-push the image as part of the build process, you can do so manually by running: docker push <full image name>, where <full-image-name> includes the account name, repo name ("conforma-demo") and tag.

Example:
docker push msupplyfoundation/conforma:build-v0.2.0-7_2022-04-07_ee35c8

Test locally

When the build (above) completes, it should print a command for running the new image locally, which you can copy and paste. It will be something like:

yarn docker_run msupplyfoundation/conforma:build-v0.2.0-7_2022-04-07_ee35c8

You'll need to make sure you have the SMTP_PASSWORD in your local .env file.

To see the actual Docker commands that are constructed, please inspect the file /docker/run.sh

Log in to the demo server with ssh

  • Get key file from Bitwarden (Conforma demo (5 instances) server) and save locally (e.g. in ~/Documents/private/conformakey.pem)
  • SSH login to server: 
    export KEY_LOC='/Users/<you>/Documents/private/conformakey.pem' (or your local location)
sudo ssh -i $KEY_LOC ubuntu@conforma-demo.msupply.org
  • Alternatively, log in to any specific country server
  • View a list of currently running images:
    sudo docker container ls
  • Stop all instances:
    sudo docker stop <hashes… >
  • Pull image from docker hub:
    sudo docker pull <full-image-name> Example: sudo docker pull msupplyfoundation/conforma:build-v0.2.0-7_2022-04-07_ee35c8

If the server is already configured, you can skip to docker-compose

Move files/folder to/from instance

Follow this process whenever the docker-compose or nginx configs are updated (./docker/demo_server/docker-compose.yml)

Copy script folder demo server scripts

Note: You are sending local changes to the server

cd conforma-server/docker
scp -r -i $KEY_LOC ./demo_server ubuntu@conforma-demo.msupply.org:/home/ubuntu/

Save backup of nginx config from demo server to local

cd conforma-server/docker
scp -i $KEY_LOC ubuntu@conforma-demo.msupply.org:/etc/nginx/sites-enabled/default ./demo_server/nginx_config

Upload nginx config back to demo server

Option 1: Multi-instances server

Note The configuration of nginx uses the certificate of conforma-demo.msupply.org domain (following other steps of this setup) if needed for a new server, replace with new domain.

# cannot directly replace default config, need to do it as sudo, so from within docker instance
sudo mv demo_server/nginx_config/default /etc/nginx/sites-enabled/

Option 2: Single-instance server

Note: First you need to have a domain configured and have been through the steps to Install SSL Certificate with certbot as described in the New server instance guide

The file is configured to forward incoming requests to port 50000 to internal port 80000 for Conforma App and requests to port 50001 to internal port 80001 for Grafana both running as docker containers.

You will need to open the file in demo_server/nginx_config_single/default and edit where is showing as <host-domain> replacing with the actual domain name (as configured in the SSL certificate).

sudo mv demo_server/nginx_config_single/default /etc/nginx/sites-enabled/

NGINX

Everything should be configured via default config. Cert bot was installed and should auto update certs (https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx).

After changing config as per above, run

sudo service nginx restart

Logs are in /var/log/nginx

docker-compose

docker-compose allows us to map specific container folders to actual folders on the host system (volumes) so that data can be persisted between container restarts (such as when doing an app upgrade). See the `docker-compose.yml' file to see the volume mappings we are using.

Volumes are created automatically the first time each docker-compose container instance is launched.

Docker stores these volumes in its internal data folder (/var/lib/docker/volumes in the case of our demo server), so it is a good idea to make a symlink to this folder in a more accessible place (such as in the demo_server folder where the docker-compose.yml file is). To make a symlink:

ln -s <source-folder> <target-folder>

E.g. (as on our demo server):
ln -s /var/lib/docker/volumes ~/demo_server/docker_data_volumes

Launching instances with docker-compose

For each instance, docker-compose is expecting a set of environment variables, some of which are contained within an .env file, and some included on the launch command.

  • In an .env file:
    • SMTP_PASSWORD -- password for the email server specified in the "sendNotification" action
    • WEB_HOST -- full host name as will be shown in the application URL (including port), e.g. https://conforma-demo.msupply.org:50004
    • BACKUPS_FOLDER (optional, default: ~/backups) -- path on the host system where the internal "backups" folder should be mapped to (optional -- default is the default volumes location). An appropriate location would be a folder that is synced to a cloud backup service. (See Backups for more info.)
    • BACKUPS_PASSWORD (optional) -- password for encrypting the backup archives (AES-encrypted .zip files) (if no password provided, the backups will be unencrypted .zip files)
  • As part of the launch command (not included in .env file as they will change every time):
    • PORT_APP: -- the Http port the Conforma server will listen on (recommend start with 8000 and increase by 2 for each additional instance)
    • PORT_DASH: -- the Http port the Grafana server will listen on (recommend start with 8001 and increase by 2 for each additional instance)
    • TAG -- the name of the tag you're about to launch
    • JWT_SECRET -- private key for generating and verifying JWT tokens. Should be a strong, randomly generated string.

Then, for each instance, run the following launch commands (you can either export the env vars or include them in the command):

Option 1:

export PORT_APP=8000
export PORT_DASH=8001
export TAG='build-v0.5.6-76_2023-08-09_802505'
export JWT_SECRET=$(openssl rand -hex 64) #one possible way to generate

sudo -E docker compose --project-name 'conforma-on-8000' up -d
# Change project name 'conforma-on-8000' for each instance

Option 1:

PORT_APP=8000 PORT_DASH=8001 TAG='build-v0.5.6-76_2023-08-09_802505' JWT_SECRET=<long-secret> sudo -E docker compose --project-name 'conforma-on-8000' up -d

-d is for detached, if you want to see all output then start without -d

Note the above commands must be run within directory where compose.yml is contained. Typically this is /demo_server

This will either launch or relaunch the server at the port specified in the PORT_APP

Stop instances and reset volumes

Stop a particular docker-compose instance:

PORT_APP=8000 PORT_DASH=8001 sudo -E docker-compose --project-name 'conforma-on-8000' down

The data volumes will persist, so next time that instance is launched (with docker-compose), data will be unchanged.

To reset (i.e. delete) a volume, use the following command (when instance is stopped):

sudo docker volume rm <name_of_volume>
(See the volumes folder to get the exact name of each one)

CAUTION: Removing a volume will delete all the data on that instance. Ensure you have exported a snapshot before removing a volume!

To remove all currently unused volumes at once:

sudo docker volume prune

View logs

# don't need bash inside container for this
sudo docker exec -ti conforma-on-8000_app_1 cat /var/log/conforma/server.log
sudo docker exec -ti conforma-on-8000_app_1 cat /var/log/conforma/graphile.log

Other image/container commands

  • List local images

sudo docker images

  • Stop a specific container

sudo docker stop <name-or-container-id>

  • Remove a container or its image:

sudo docker rm <name-or-id>

  • Remove all containers

sudo docker rm $(sudo docker ps -a -q)

  • Remove all images

sudo docker rmi -f $(sudo docker images -a -q)

Access command line within container:

You can access the command line of a particular container instance with the following:

sudo docker exec -ti <name-or-container-id> /bin/bash

From there the following commands might be useful:

  • view environment variables: printenv
  • check the server log: tail -n 100 /var/log/conforma/server.log

Copy the log file to your local environment:

  1. Log in to the server using terminal

  2. Copy the log to an accessible place (in the server) i.e. if getting log for instance 5000:

    sudo cp /var/lib/docker/volumes/conforma-on-8000_logs/_data/server.log demo_server/2022-11-15.log

  3. Now exit the server exit and on your local machine open the terminal

  4. Navigate to a folder where you would like to save the log, run (some similar to this):

    scp -i $KEY_LOC ubuntu@conforma-demo.msupply.org:/home/ubuntu/demo_server/2022-11-15.log .
  • KEY_LOC is an environment variable for the server public key :)
  • And the . means it will create a file with the same file name on the current folder

Managing CloudVPS hosted servers

Conforma live servers are usually hosted on (CloudVPS), an (openstack) server provider.

Changing CloudVPS server settings

Prior to changing any settings on CloudVPS, it is essential to take a backup snapshot of the live server, and ensure back ups are logging correctly to dropbox or wherever you are keeping your backups.

  1. Log in to (CloudVPS). Credentials are available through Bitwarden.
  2. Access all instances through compute / instances
  3. Search for the relevant instance ie “conforma”
  4. Click “resize instance” from dropdown
  5. Select new flavour package
  6. Upon set up, conforma docker image will need environment variables set up again, and run, using the commands above. Images and snapshots will persist when a server's capacity is changed.

mSupply logo

Powered by mSupply

Clone this wiki locally