Publish image to container registries #163

arledesma · 2021-05-10T05:55:21Z

In order to enable quick development and deployment, without requiring an hour long build, this PR will enable publishing to a container registry via configurations setup in the github repository secrets.

When running through a proof of concept with the current solution we found that there was ambiguity due to completely different solutions for local vs production.

This PR attempts to alleviate some of that ambiguity by:

consolidating the compose/local and compose/production directories into only compose.
declaring the base services directly within docker-compose.yml
- local.yml -> docker-compose.override.yml which is automatically applied
  - docker-compose up -d automatically starts local services
  - docker-compose up -d --build builds and starts local services (should only be required for changes to requirements.txt)
- docker-compose.dev.yml extends docker-compose.yml for debugging
  - docker-compose -f docker-compose.yml -f production.yml up -d automatically starts local services with debugging
  - docker-compose -f docker-compose.yml -f production.yml up -d --build builds and starts local services (should only be required for changes to requirements.txt)
- production.yml extends docker-compose.yml
  - docker-compose -f docker-compose.yml -f production.yml pull pulls the latest referenced images
  - docker-compose -f docker-compose.yml -f production.yml up -d starts production services, including nginx
  - docker-compose -f docker-compose.yml -f production.yml up -d --force-recreate restarts production services
  - this should not use a build but instead point at a stable image that has been published (currently set to 2.2)
moves the django Dockerfile into a multi-stage build, utilizing a STAGE build argument to determine which stages will be executed.
- Typically requirements.txt will not be updated very often. When it is required then the build tools required to enable for it to be successful, as well as many artifacts associated with the build, should not be carried in a layer.
- This has decreased the uncompressed image size from >1GB to ~550MB (compressed ~160MB)
updated .gitattributes to help ensure that files are committed with the correct LF EOL. While this does not explicitly ensure that windows users will never see odd behavior it will help protect the files within the repository.

github action

A github action workflow has been added that will build local branches and push to GitHub Container Registry and/or Docker Hub
examples:

The meta step will generate a number of labels, including semantic version/latest/edge and branch (for your repository)
The semantic version requires a valid semantic version as a tag to be pushed. Multiple labels will be produced from that in order to enable rolling images. e.g. 2, 2.2, and 2.2.1 could all point at the same image. If 2.2.2 is subsequently pushed then both 2 and 2.2 will then point at the same image as 2.2.2 while 2.2.1 remains in place. When a new tag is pushed then that image will replace the latest label. This will hopefully enable users to choose how much risk they are willing to accept during their production deployments by explicitly referencing a specific label.
Branch labels can also be used, such as ghostwriter:master or ghostwriter:edge where edge is the latest commit on the default branch.

branch/tag	label
master	master
master	edge
v2.3.4	2
v2.3.4	2.3
v2.3.4	2.3.4
v2.3.4	latest

A build is also scheduled to occur at 03:00 on the 1st and 15th days of the month. These should produce an image with a label in the format of YYYYMMDD. These builds can be useful to help ensure that any underlying system packages are updated within the container to assist when there may be lulls in development work within this repository.

Additionally this further addresses #143 by explicitly configuring the nginx service volumes of media and staticfiles to set read_only and nocopy:

read_only: true
volume:
  nocopy: true

https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes

read_only: flag to set the volume as read-only

volume: configure additional volume options

nocopy: flag to disable copying of data from a container when a volume is created

Add Github Action for Container build and push Limit number of container layers and reduce size Update COPY chmod to use octal Stop copying DOCS directory into container Reduce number of production container layers Remove linux/arm64 container platform Convert django container to multi-stage build Reduction of image from ~1GB to <550MB Conslidate production/local docker-compose/Dockerfile Move compose/production/* into parent compose/ folder Remove defunct compose/local folder Remove sharing lock from cache mount on RUN The cache was empty when using a `locked` sharing type. Reverting to the default sharing of `shared` as the build should not be affected with the current workflow. https://github.com/moby/buildkit/blob/f2a6e83adcb0295099870489b76d3ce74d6f7f42/frontend/dockerfile/docs/syntax.md#run---mounttypecache > This mount type allows the build container to cache directories for compilers and package managers. |Option |Description| |---------------------|-----------| |`sharing` | One of `shared`, `private`, or `locked`. Defaults to `shared`. A `shared` cache mount can be used concurrently by multiple writers. `private` creates a new mount if there are multiple writers. `locked` pauses the second writer until the first one releases the mount.| Update workflow to run on 'master' branch Limit GITHUB_TOKEN permissions for job Only push the container on push and schedule events ghaction-docker-meta action moved from crazy-max to docker org https://github.com/docker/metadata-action/releases/tag/v3.0.0 Use bind mount rather than cache for wheels cache is not guaranteed. build the wheels and allow pip to cache during the process. bind mount the wheel-dir when installing in django stage Consolidate start commands Remove extraneous instructions from Dockerfile - no need to install the requirements in the build stage - only build the wheels - gecos is for storing metadata about a user (full name, phone number...) - copy of requirements from build is now handled transparently through a bind mount without requiring the additional layer - du of /tmp/wheels was only for debugging the cache mount which is now a bind mount - /tmp/requirements is a bind mount so it does not need to be removed from the stage Cache first stage of multi-stage build By default the mode is set to `min`, which only exports layers to the cache in the final build stage. We want to cache the first stage in order to not always build wheels. Specify ghostwriter:2.2 image in production.yml Add missing EOF newline to docker components Update to default compose in run-unit-tests github workflow job local.yml was replaced with docker-compose.override.yml which is automatically applied when no other compose files are specified

arledesma · 2021-05-12T02:45:13Z

@chrismaddalena let me know if there is anything else that I can expand on or if you need anything changed.

arledesma · 2021-08-18T04:18:59Z

@chrismaddalena I have gone ahead and rebased this branch based on what is currently in master.

We would like to be able to deploy an official image of Ghostwriter in the near future, so is there anything else that you would like to see in order to get this merged?

chrismaddalena · 2021-08-19T21:40:41Z

@arledesma Hey! Life has been busy so I haven't had a chance to give this a close look. Getting images on DockerHub is high on my list, so I'll take a look as soon as I can.

bman46 · 2023-11-07T00:10:14Z

Any updates for this? I would like to build a helm chart for this app and I need a public docker image to do that.

arledesma force-pushed the feature/push-to-container-registry branch 8 times, most recently from a8f733b to b3a2e47 Compare May 11, 2021 05:40

chrismaddalena mentioned this pull request May 11, 2021

Bugfix - Production Django Container Restart Loop #143

Closed

arledesma force-pushed the feature/push-to-container-registry branch 11 times, most recently from 2c1f377 to e574436 Compare May 12, 2021 02:05

arledesma marked this pull request as ready for review May 12, 2021 02:43

arledesma changed the title ~~WIP: Publish image to container registries~~ Publish image to container registries May 12, 2021

arledesma force-pushed the feature/push-to-container-registry branch from 3fb8cdf to 1c047d5 Compare August 18, 2021 04:17

arledesma force-pushed the feature/push-to-container-registry branch from 1c047d5 to 0172a9e Compare June 14, 2022 21:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Publish image to container registries #163

Publish image to container registries #163

arledesma commented May 10, 2021 •

edited

arledesma commented May 12, 2021

arledesma commented Aug 18, 2021

chrismaddalena commented Aug 19, 2021

bman46 commented Nov 7, 2023

Publish image to container registries #163

Are you sure you want to change the base?

Publish image to container registries #163

Conversation

arledesma commented May 10, 2021 • edited

github action

arledesma commented May 12, 2021

arledesma commented Aug 18, 2021

chrismaddalena commented Aug 19, 2021

bman46 commented Nov 7, 2023

arledesma commented May 10, 2021 •

edited