diff --git a/docs/source/backup.md b/docs/source/backup.md index 6f8451048..9a4015713 100644 --- a/docs/source/backup.md +++ b/docs/source/backup.md @@ -14,14 +14,19 @@ The WAL-G backup setup is integrated into the Postgres Docker image. It will do WAL-G is installed alongside Postgres. See the Dockerfile of Postgres (`postgres/Dockerfile`) for details. In the docker-compose postgres command (`docker-compose.yml`) archive parameter of postgres are set needed to make archives. -## Restore setup +### Configuration -The WAL-G restore setup is realized in a dedicated Docker image (`postgres/Dockerfile-restore_backup`). It will create a new Postgres database cluster, fetch latest backup using `wal-g backup-fetch` and create a `recovery.conf` file in the new cluster during Docker build. `recovery.conf` is used by Postgres during first start to get the `restore_command`. Again the exact commands are to be found in `postgres/wal-g/restore_command.sh`. During first start Postgres will get WALs from backup server and restore the database. +To store backups in Google Cloud Storage, WAL-G requires that this variable is set: `WALG_GS_PREFIX` to specify where to store backups (eg. `gs://x4m-test-bucket/walg-folder`). +Please add this to the `.env` file at the root of MapSwipe Back-end (See `.example-env` for environment variables which have to be set) +WAL-G determines Google Cloud credentials using application-default credentials like other GCP tools. Get a Service Account Key file (`serviceAccountKey.json`) for your Google Cloud Storage (See [Google Cloud Docs](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)) and save it to `postgres/serviceAccountKey.json`. -## Configuration -To store backups in Google Cloud Storage, WAL-G requires that this variable be set: `WALG_GS_PREFIX` to specify where to store backups (eg. `gs://x4m-test-bucket/walg-folder`). -Please add this to your `.env` file at the root of MapSwipe Back-end (See `.example-env` for environment variables wich needs to be set) +## Restore setup -WAL-G determines Google Cloud credentials using application-default credentials like other GCP tools. Get a Service Account Key file (`serviceAccountKey.json`) for your Google Cloud Storage (See [Google Cloud Docs](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)) and save it to `postgres/serviceAccountKey.json`. +The WAL-G restore setup is realized in a dedicated Docker image (`postgres/recovery/Dockerfile`). The entrypoint is the `ini_recovery.sh` srcipt. This script will create a new Postgres database cluster, fetch latest backup using `wal-g backup-fetch` and create a `recovery.conf` file in the new cluster. `recovery.conf` is used by Postgres during first start to get the `restore_command`. Again the exact commands are to be found in `postgres/recovery/restore_command.sh`. During first start Postgres will get WALs from backup server and restore the database. + + +### Configuration + +> The same configuration as for the backup setup is requiered. Except the Service Account Key has to be stored at `postgres/recovery/serviceAccountKey`. diff --git a/docs/source/contributing.md b/docs/source/contributing.md index 00d8b498c..5814b692d 100644 --- a/docs/source/contributing.md +++ b/docs/source/contributing.md @@ -1,12 +1,49 @@ # Contributing -This document is a work progress. It should describe how to contribute (code or issues) to the MapSwipe back-end. -To contribute to the MapSwipe back-end please create dedicated feature branches. +## Clone from GitHub ```bash +git clone https://github.com/mapswipe/python-mapswipe-workers.git +cd python-mapswipe-workers +git checkout dev +``` + + +## Install MapSwipe Workers + +Create a Python virtual environment and activate it. Install MapSwipe Workers using pip: + +```bash +python -m venv venv +source venv/bin/activate +pip install --editable . +``` + + +## Feature Branch + +To contribute to the MapSwipe back-end please create dedicated feature branches from dev: + +```bash +git checkout dev git checkout -b featureA git commit -am 'add new project type' git push -u origin featureA -git request-pull origin/master featureA +git request-pull origin/dev featureA +``` + +> Note: If a bug in production (master branch) needs fixing before a new versions of MapSwipe Workers gets released (merging dev into master branch), a hotfix branch should be created. In the hotfix branch the bug should be fixed and then merged back with master and also dev. + + +## Style Guide + +This project uses [black](https://github.com/psf/black) and [flake8](https://gitlab.com/pycqa/flake8) to achieve a unified style. + +Use [pre-commit](https://pre-commit.com/) to run `black` and `flake8` prior to any git commit. `pre-commit`, `black` and `flake8` should already be installed in your virtual environment since they are listed in `requirements.txt`. To setup pre-commit simply run: + ``` +pre-commit install +``` + +From now on `black` and `flake8` should run automatically whenever `git commit` is executed.