Share config between containers...

[rliebling]

Would like your input on this:

I have a django app deployed with crane. So, one container runs the django app and another the webserver (nginx), etc. The django app has various environment variables specified in the crane manifest file. Sometimes, hopefully in development rather than production, someone has a need to execute various tasks associated with the app, like manage.py shell or manage.py schemamigration to create a new migration, etc. These commands can be run in a separate container, but should have the same environment as the appserver itself. Three possible approaches are:

1. The "do nothing" approach would have a separate crane manifest file with most everything duplicated (especially the environment) and just specify whatever entrypoint or command you want. I don't like this because when something needed to be changed, the user must remember to change in both manifests.
2. Another approach that might almost "just work" would be to define in the same manifest file both the app container and another container for the command (maybe just a bash shell where the user can then manually run whatever manage.py commands they want). But, yaml has a way to share a node (search for 'repeated nodes' in http://jessenoller.com/blog/2009/04/13/yaml-aint-markup-language-completely-different, for example). So, could use that and I expect the go yaml parser would handle it. Then, the only thing to add to crane (I think) would be a feature to run or not run a specific container from within the manifest. That is, normally you'd want to bring up the appserver and the webserver, but not the shell. But, once in a while you'd want to just run the shell.
3. Another approach would be to allow templating within the manifest. IN this way you might allow environment variables to alter the resulting manifest for crane, changing the command or entrypoint, for example, and skipping certain containers. This is probably more powerful in general, but also perhaps ugly and more difficult for the use case i describe.

I'm somewhat inclined toward 2, even though i wouldn't be surprised to want templating at some point anyway :)

Thoughts/suggestions?

Oh, another use case that's occurred to me, but i don't yet need, is I expect to want to use crane to build images and push them to a docker registry. I can easily imagine wanting to choose between multiple registries without having to edit the manifest file (ie to change the tag given to the image). One registry might be local to my computer, another might be for the team. Potentially this could be done using an environment variable in the value of the image name. Adding a push command to crane would certainly be pretty simple. Anyway, i think this use case is handled pretty easily without affecting the choices above.

[michaelsauter]

Sorry for the late reply, I'm super-busy at the moment ...

Interesting thoughts! I have thought about adding the concept of environments (e.g. run containers with different parameters in different contexts such as development or production). Maybe that would address some of the issues that you face. For example, you could have a "dev" environment, which brings up the container to execute the tasks.

Overall, I'd really like to keep Crane as simple as possible. Templating sounds tempting, but I think it's too much. At least not right now with very little use cases that definitely need it. I'm very sceptical of early abstractions/features and think that most of the time, repeating things is not as bad as most devs think. It's cheap, and only when it really hurts it makes sense to abstract it.

So. Overall, I like 1) and 2) most. Not sure if the Go YAML parser can handle it, but if it can, that's really nice! Didn't know YAML can do that.
For 2), I'd think something like this would help:

"containers": {
  "one": {},
  "two": {},
  "three": {}
},
"environments": {
  "dev": ["one", "two", "three"],
  "prod": ["one", "two"]
}

Then, we could add a flag to crane to bring up just one environment, e.g. crane lift --env prod.

What do you think?

Also, you're idea to push containers to a registry sounds great! I've created a new issue for it: #19

[michaelsauter]

I think this is the way to go, with a minor modification. So the final manifest would look like this:

"containers": [
  { "name": "one" },
  { "name": "two" },
  { "name": "three" }
],
"groups": {
  "dev": ["one", "two", "three"],
  "prod": ["one", "two"],
}

crane lift would bring up all containers, crane lift --group prod just one and two. Also every container should be a group automatically, so crane lift --group one should work as well.

[michaelsauter]

The groups are implemented in fe60f4d.