feat: add docker-compose stack

[paddatrapper]

Description

Adds Dockerfiles and documentation around them

• Shared
• API client
• Analyzer
• API
• Legacy
• Worker
• Playout
• Liquidsoap
• Documentation
• CI push to Docker image repos

Fixes: #949
Fixes: #551
Fixes: #624

This is a new feature: yes

Do the changes in this PR implement a new feature?

I have updated the documentation to reflect these changes: TODO. Needs deployment and dev notes

Testing Notes

What I did:

Built and tested Docker images

How you can replicate my testing:

Follow the Docker build documentation (TODO)

[jooola]

Hey, some early review hope you don't mind.

Awesome that you started doing this, it was also my intent you beat me to it.

I left a few comments. And maybe we should pin the python release version as well.

And I would not recommend using bullseye, I mean we need to move forward and fix the codebase, but I reverted my migration to bullseye in production as I was getting way too much bugs.

[paddatrapper]

Building individual docker images is tricky without #1504 because the build context is each app. So inter-dependencies (e.g. on api_client) are not supported. They can only be installed from the web and not from the local repo, making pypi the logical choice. It does mean that changes to shared/api_client require a version bump before the changes can be used in any application. This is because building will always use the Pypi version, so new changes need pushing there before being used.

For example, this blocks building a Docker image for playout, as it requires the api_client package to be installed to run

[jooola]

I would recommend importing a wheel file from a build stage over fetching packages from pypi.

I used to release some docker images like this, but by the time the pip package was available on pypi, the published docker image was using the previous version. Also this would not allow use to build images that haven't been published yet. For instance having a image from master for the testing/demo feature would be infeasible.

What I propose is to create a api_client and shared dockerfile only with a build stage, run those before we build the apps, and import the wheels from there.

May I push some ideas to this branch ?

[jooola]

Pushed a extra commit with some idea, feel free to prune the commit if you don't want it.

[jooola]

I rebased this branch.

[jooola]

Rebased the branch onto main.

[paddatrapper]

@jooola please make autogenerating requirements a seperate PR

[jooola]

Rebased on the main branch.

@paddatrapper Please tell if I should have my own branch if I am messing with your work.

[paddatrapper]

No, it's fine. I want to go through and clean this up hopefully this week so it is ready for testing

[paddatrapper]

Where do we want to publish docker images? GitHub container repo and Docker Hub?

[jooola]

I lean towards ghcr

[jooola]

So I've been playing and developing with this for the past days, and it is quite pleasing to use ! Everything is working as expected, and a "production" demo is available in docker/example (since no image are published yet, you have to build the images first and override the LIBRETIME_VERSION=main to run them).

This is obviously not bullet proof yet but haven't had any bug so far. This can be considered as the work in progress I was referring some days ago, as we will have to fix bugs, improve the documentation and maybe change installation procedures.

With regards to inter container dependencies, I am not yet sure if we want to wait for others services apart from the database and maybe rabbitmq. For now we have the dependencies set, maybe I'll remove them in the future if it improves the startup time, and they are properly handled.

With that said, I think we can review merge this !

[paddatrapper]

Awesome! Thanks for all the hard work. I'll try test and review on Monday

[jooola]

Rebased onto the main branch.

[paddatrapper]

/website-preview

[github-actions]

🚀 Website preview deployment succeeded!

Website preview: https://libretime.github.io/pr-1471/
New docs preview: https://libretime.github.io/pr-1471/docs/next/
Workflow: https://github.com/libretime/libretime/actions/runs/3051410854

[paddatrapper]

I'm getting the following error running from the root of the project:

$ docker-compose run --rm api libretime-api migrate
ERROR: The Compose file './docker-compose.override.yml' is invalid because:
services.api.environment.LIBRETIME_DEBUG contains true, which is an invalid type, it should be a string, number, or a null

[jooola]

I'm getting the following error running from the root of the project:

$ docker-compose run --rm api libretime-api migrate
ERROR: The Compose file './docker-compose.override.yml' is invalid because:
services.api.environment.LIBRETIME_DEBUG contains true, which is an invalid type, it should be a string, number, or a null

Which version of docker-compose are you using ?

[paddatrapper]

Fixed by quoting the variable. Running docker-compose 1.29.2

[jooola]

Fixed by quoting the variable. Running docker-compose 1.29.2

Ok makes sense, I've only been working with docker-compose 2.x.

[jooola]

/website-preview

[paddatrapper]

LGTM - tested and seems to work for me now. I'm excited to try this within k8s once we have built images and see how things go

[jooola]

LGTM - tested and seems to work for me now. I'm excited to try this within k8s once we have built images and see how things go

Awesome ! As we discussed earlier, I expect this to be somewhat of a work in progress. I hope that the feedback from the community will help use iron out the docker setup.

I'll merge in that case.

[jooola]

Oh but now that you mention it, should we publish a image for the main branch ? Or only tagged release ? I only considered the tagged releases for now.

EDIT; merging, this can be tweaking in another PR