Secret management

[ScottG489]

Currently secrets are passed in as input to containers being run. This is fine for most cases since we want it to be the callers responsibility to manage secrets.

However, there still may be some use cases where we want to be able to initialize secrets so a job can access them when run.

For instance, take a caller that needs to run a particular job. This job requires credentials to a 3rd-party to carry out it's tasks. However, the caller may not have permission to this third party.

Unfortunately, I think this will require some sort of state. Statelessness is something we want to maintain whenever possible. However, I don't think it can be entirely avoided if we want this functionality. A good compromise may be to have this state entirely managed by the docker application if possible.

So far I have looked into

• Docker secrets
• Docker volumes

On first pass at least, docker secrets don't seem to be an option. They seem only able to be used with docker swarm and swarm doesn't seem to mesh well with our use case. It seems a lot more focused on services. And since our jobs are meant to terminate it seemed like we're using it for something it isn't meant for.

Now, I am currently looking into using docker volumes. The idea would be to allow a secret to be registered by storing it in a volume and for a specified image. Then when the image is run it would have access to this secret. I believe this should be safe enough because when you create the secret you specify which image(s) have access to it by name.

However, one big issue arises from the fact that all images that are run are privileged and can control the host dockerd (since docker.sock is mounted on all run images). This means that they have full access to run anything on the host dockerd and thus would have access to any volume and view anything stored in these volumes, such as a secret. Since containers are allowed to run docker freely there is little that can be done about this right now.

I think a longer term solution would be to not run all images with docker.sock mounted. This would be safer in general since doing this in the first place is a bad idea for security reasons. However, this will also break current builds that expect this to be allowed and do things like building docker images within their builds.

One solution to this would be to require builds be more "modular" and instead call specific docker images like the "docker-build-push" image to handle docker-related tasks.

For instance, in a build, an application could build it's binary and put it on a docker named volume (unrelated to secrets) then quit. Then an approved/privileged image like docker-build-push could mount that volume and run the docker commands that it needs to. This would probably require some kind of whitelist on the server so that only specified images would be allowed to run as privileged.

[ScottG489]

Closed by the following:

67374e7
36819a9

Note that this introduces the security vulnerability discussed in the original comment and in the issue for fixing it #11