Conversation
This modernizes the use of docker-compose (avoiding the need for links and using structured labels) and adds a "config" container to listen for Docker events, update the worker list file, and SIGHUP the master to reload configuration files. This removes the need for hardcoding the number of workers, worker ports, links, and the reload-workers shell script.
|
Thank you, from an initial glance this looks awesome! I will be testing this out and giving feedback next week! |
| config: | ||
| container_name: 'citus_config' | ||
| image: 'citusdata/workerlist-gen:0.9.0' | ||
| volumes: ['/var/run/docker.sock:/tmp/docker.sock'] |
There was a problem hiding this comment.
What does this enable us to do? Is it to be able to use the docker API from within the container?
There was a problem hiding this comment.
Yes.
The citusdata/workerlist-gen container uses a program called docker-gen to listen to docker events and regenerate certain files after certain events. See the top-level description on this pull request for links. From docker-gen's documentation:
There are three common ways to run docker-gen:
- on the host
- bundled in a container with another application
- separate standalone containers
I'm going with option three, which is to run docker-gen in a container with access to (a) the docker socket (to listen to the API), and (b) a shared volume with the Citus worker list file (so it can update the worker lists).
docker-gen has an option that tells it to send a SIGHUP to another container after a template has been regenerated. This is how we can signal Citus to reload config after a docker-compose scale action.
|
Hey @macobo, I realized I never followed up here with the fact that we've released our own Docker image as part of our open-source/5.0 release. It's here if you have any interest in trying it out (compose and Docker Cloud files are included in the repo we use to build the image). |
2 participants
Hey Heap,
I've been working on official Citus Docker images and configuration. While most of that is still under wraps, I thought you all might be interested in the Docker configuration changes I made. Essentially, I found the
/etc/hostssolution limiting (especially in conjunction withdocker-compose scale), and wanted to use the new networking features (which no longer requirelinksetup). I found that usingdocker-genlet me do this pretty smoothly (itSIGHUPs a container when containers enter or leave).Tell me what you guys think. I'll also send an email as followup. Your CitusDB 4 Docker work is great and I've already come across others using and extending it, so I just wanted to contribute back to what you've started!
Description of changes