-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update docker + docker compose for dev and prod #1021
Conversation
just a thought but maybe we should use 1 compose file with profiles instead of multiple compose files for prod and dev as i think being able to keep the compose.override file separate for admins to make changes to the compose.yml instead of editing it directly while also being able to keep up with git changes |
@CocoPoops that doesn't seem like the right feature to use, unfortunately.
Additionally, if you look at We could use a justfile to have a |
@LoveIsGrief hopefully this explains what i meant better For example, It would allow us to create php-prod and php-dev services in one compose.yml, along with services, and then assign them to corresponding profiles or both if no changes are between prod and dev. This would allow administrators to override settings for either production or development services independently through the compose.override.yml file. This flexibility is particularly useful in scenarios like mine, where I use Traefik as a reverse proxy configured via Docker labels. By organizing the Compose files this way, it would be much easier to switch between production and development environments for debugging purposes without the risk of exposing sensitive debug secrets. |
I think it would also be easier that way if for some reason we need to change the compose file down the line as we should be recommending admins only change things via the compose.override file |
@LoveIsGrief Could you fork my branch and make an example? For your scenario, you could use compose.override.yml include:
- compose.prod.yml
services:
traefik: ... |
ill just do this as i just tested using the compose profiles and it doesnt really seem worth it as it kinda makes it a pain to maintain if anyone wants do that include method for overriding
|
Thank you for testing!
I think I know where the issue lies.
When making the image compatible with other reverse proxies, I copy the public/ folder to a volume shared between the php service and the reverse proxy. Uploads probably happen in the php service and end up in the php container but not on the shared volume.
I think that's the issue and I'll try to find a solution to that this weekend.
Again, thanks for testing!
|
Press "Ready for review" (and remove "WIP" from the title) if you think we should review. |
@melroy89 I finally have it in a state where the code should be reviewable. I don't know if you guys squash-merge or not. If not, once the reviews are in, I can rebase and rework the commits to get rid of the @CocoPoops I think I fixed the issue you were facing with missing media. Please see the migration guide section in the installation/docker.md docs. |
Yes, we normally squash merge the commits. Especially in this PR that is a must haha. |
container_name: mbin-db | ||
restart: unless-stopped | ||
networks: | ||
- mbin_external_network |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PHP, DB & Redis can definitely be an internal network (internal: true
). Yes, you can have multiple networks in Docker. And in Compose you can also add multiple networks to a single service (eg. messenger can both have internal and external docker networks).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 I'm not sure what advantage that would have for the php
and messenger
services. If I'm not mistaken those require outgoing access to external networks in order to download stuff and communicate with other instances. The php
service should probably be able to access direct incoming traffic. Dunno about the messenger.
The DB and redis could probably be purely internal as I don't believe they'll ever need to access external resources.
Should this be done in another PR? I feel like it could expand the scope of this PR and it might be good to have a separate PR with one concern (docker compose networking) for posterity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As fair as I know only the www service (and maybe also the rabbitmq if you enabled the management interface) needs access to the external network, the rest of all the services can do internal communication between the services.
Sure, this can also be done in a follow-up PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created a ticket so it won't be forgotten.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I now get:
Error response from daemon: driver failed programming external connectivity on endpoint mbin-db (c1c3e1b40be94b691d7a32acbdcde8723b9a7402b68bbb64598c5533e354b1e5): failed to bind port 127.0.0.1:5432/tcp: Error starting userland proxy: listen tcp4 127.0.0.1:5432: bind: address already in use
So this external network will interfere with any host ports that run for example PostgreSQL. Therefore, I reopened this conversation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As Bentigerolich asked for this PR and Average Dude is waiting for approval I am approving this
curl isn't installed in the image and the health check was constantly failing
/public files are copied into a common volume shared between reverse proxy and PHP-FPM service.
Otherwise the logs get filled with 'FATAL: role "root" does not exist' everytime the health is checked
After supporting arbitrary reverse proxies, media uploads were broken. They are uploaded to public/media/ by `php` service, but the previous solution just provided a copy of public/media/ at first run. New uploads weren't shared with the `www` service. Now, instead of just providing a copy, a mount is provided. To do so the strategy was changed and a lead taken out of nextcloud's book: https://github.com/nextcloud/docker/blob/65138b6d22bec1ac15e2f0f125426290640bb97a/docker-entrypoint.sh **The problem** The image had the mbin sources in /var/www/mbin. Mounting docker/storage/public into /var/www/mbin/public would replace the image's public/ with a (hitherto) empty directory. **The solution** The image has its sources in /usr/src/mbin and on startup, syncs that into the new mount /var/www/mbin using `rsync`. In this manner, previous uploads are preserved and source changes update /var/www/mbin.
Separate images are now available because mbin isn't solely bound to caddy and can be used with any reverse proxy.
These come due to the refactoring: - `docker/compose.yml` --> `/compose.yml` - support for arbitrary reverse proxies without running caddy - smaller docker image - allow overrides without changing versioned files
Co-authored-by: Melroy van den Berg <melroy@melroy.org>
Co-authored-by: Melroy van den Berg <melroy@melroy.org>
It wouldn't start without the file and I realised it had been there the entire time when testing the documentation. @melroy tested it without and discovered the bug
This follows what is done for the production guide, which has 2 guides/files as well.
I stopped postgresql locally, so it now starts the docker images. Going to http://localhost:8080 gives me:
|
First of all, I apologize for the size of the PR 🙏, but I think it would've been difficult to break it down.
This is a PR to rework the docker setup for dev and prod. The highlights are below, but basically the setup was not working well for development (not clear how to set it up, which files were necessary to be where, which env vars were important, updating code locally didn't reflect in the docker environment, etc.).
Now, developers should be able to spin up a development environment with minimal effort with docker installed, see their changes show up on reloads, and even use a debugger (XDebug).
The changes in the dev setup led to changes in production setup as well, though they should be minimal. Most importantly
caddy
was pulled out of the image to break the tight coupling and allow any reverse proxy with fast CGI to sit in front of mbin.Highlights
Features
Details
host.docker.internal
doesn't exist on linux because it's not running docker in a VM. Add following block tophp
service.Bug fixes
FATAL: role "root" does not exist
indb
service created by its healthcheckTODO
sudo rm -rf docker/storage
.gitignore
out of storage otherwise resetting still creates a commit by deleting.gitignore
docker/storage
needs to be writable tombin
user!sudo chmod 777 docker/storage/*
Abandoned tasks for later PRs
MESSENGER_TRANSPORT_DSN=doctrine://default
as documented doesn't work