Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add "secret" as allowed build args #36443
Signed-off-by: Manabu Sakai firstname.lastname@example.org
- What I did
Added build-args to pass secrets variable when
I want to improve this.
- How I did it
- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)
Hi @tonistiigi - Thank you very much for maintaining moby/docker!
However, I still find this change to be useful enough to get merged.
Even though this isn't very intuitive at first glance, I'd like to emphasize that this is a GREAT step forward for the secret management
I'm currently using something like
Obviously, this isn't a perfect solution to the secret management problem as:
However, that's much better than unexpectedly exposing it via docker image metadata/layers. I believe this method should be used more widely wherever possible.
This PR just solves the 3rd issue with a minimum change, without any unwanted side-effects(I suppose so?).
Full disclaimer: I'm a colleague of @manabusakai
I wished #30637 to get merged in the past, but AFAICS it didn't make it due to the design issue.
If we can't have something like that in docker before builtkit goes GA, and not wishing to switch to alternative builders like support for secrets:
this change is understandable to me.
Edit: I'm not saying any of those projects are bad. Not at all, they all are awesome. I'm just saying they all have different sweet-spots, not drop-in replacements of docker! And we're already docker users.
It is finally time for us to save people from accidentally leaking build creds, or giving up build caching, or switching to another tools only for secret management!
left a comment
Programs that are totally unrelated to Docker/Moby may use
So, at least, the environment variable name must be changed to an unique one. e.g.
However, I don't think using environment variables for secret management is correct.
Also, potentially (although unlikely), future version of Docker (or other Dockerfile-compatible tools) MAY capture the environment variable set on completion on the each of the
For a while, you may want to use
Good point! I agree with your suggested change. @manabusakai WDYT ?
@AkihiroSuda Thanks for the review!
I'm not really arguing here, but envvars won't be leaked without explicit actions. If you
Sounds awesome towards more reproducible builds! Keep up the great work!
You may already know but the envvars and build-args are two different things.
Would this be your suggested change?
The approach taken in this PR is much simpler and just works without any NEW drawbacks and huge engineering efforts.
Thx! It is always nice to get to know about more workarounds.
You may already know but we have bunch of them:
Again, my point is that the approach of this change is much better than all those existing workarounds due to its simplicity.
@AkihiroSuda Please consider my above comment as-is. I have no mean to force you maintainers to merge this, AT ALL of course. That's maintainers' responsibility or freedom to do so.
I just wanted to sync up with you, as you seemed to have missed my points(to me sry)!
I'm not comfortable with this change, and I don't think this will address many use-cases. First of all, build-args were not designed for handling secret data; this was the main reason for explicitly mentioning in the docs that they should not be used for this.
While it's possible to use build-args for secrets (e.g. in a multi-stage build, as long as you take care not to have your secrets end up in the final image), they were not designed for it. The code path doesn't treat them secure, so it's generally not recommended.
Excluding a single name (
Based on the above, I'm -1 for this change, but agree that a long-term solution is still needed
First of all, I'd try to keep my secrets out of my Dockerfile if possible. So instead of:
FROM foo RUN git clone <my-private-repo> RUN do-some-things
(which could also be non-deterministic)
I have my Dockerfile as part of the repository, and do:
git glone <my-private-repo> docker build ...
If secrets are not avoidable, you can still do:
FROM foo AS stage1 RUN echo $SECRET RUN mkdir /artefacts # Compile your code, etc etc, and store the artefacts somewhere RUN echo bla > /artefacts/something # Start with a clean image FROM alpine # Copy the artefacts from the first stage COPY --from=stage1 /artefacts/something /bin/something
The image produced by the first build-stage will only be present on the server where you built the Dockerfile; that image will have your secret (at least in its
docker history dfd834940c56 --no-trunc IMAGE CREATED CREATED BY SIZE COMMENT sha256:dfd834940c56300567882be6bb3e25254c8bacfce7fcbe0b660f27c46ef371a1 Less than a second ago |1 SECRET=password /bin/sh -c echo bla > /artefacts/something 4B sha256:e65ffa8ecca984e9079231a26b9c68bb4b4185ac0021ad2f6864c1ebe82172d6 Less than a second ago |1 SECRET=password /bin/sh -c mkdir /artefacts 0B sha256:6922d952c6204e44e34c2eb2216bf8fa3f0377ddd2e95fb27d059add3ff7dc79 Less than a second ago |1 SECRET=password /bin/sh -c echo 0B sha256:d02eea28ed6fa01f072cef06f6bd9719fed43d21621df4c458881bfe8d74b212 Less than a second ago /bin/sh -c #(nop) ARG SECRET 0B sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353 7 weeks ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B <missing> 7 weeks ago /bin/sh -c #(nop) ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in / 4.15MB
As long as you don't push that intermediate image (and it's not a problem that the host on which you build has that image containing your secret), the secret is not distributed.
The final image (produced by the
@thaJeztah Thank you very much for the the detailed explanation!
Seems like you and I are following the same path.
I began to avoid your first pattern due to that I have to resolve library deps for my nodejs/ruby/etcd project outside of
What I wanted achieve with Dockerfile in the first place is build reproducibility.
In that regard, running e.g.
I do chain
However, hopefully, Docker multi-stage builds + ideal secret management make this chaining unnecessary.
Also note that this pattern works well with golang projects for example. That's because you tend to vendor your private library deps in SCM, rather than resolving it at build-time.
And in your second pattern - I believe you already know though - you're basically giving up layer caching for the first
I'm really curious. What would you do today if you want a maximum build reproducibility + faster docker builds without leaking secrets?
Layer caching should be the same: multi-stage builds will still use caching, assuming you build again on the same host.
@thaJeztah Thanks for clarifying!
In most of my use-cases though, docker hosts are ephemeral. So I tend to
Perhaps in your case this wan't a problem because you are happy with the long running docker host or discarding layer cache occasionally, right? (Just curious. Everyone seems to have too different use-cases and contexts when talking about this secret management + extra use-case)
My best bet after that is to close this as not "change requested" but rather rejected, and consolidate this discussion.
Thanks anyway for taking your time, and maintaining moby/docker!
@manabusakai Thanks for this PR!
Unfortunately, we'd better close it for now and continue discussion in #13490 if necessary, so that we won't trouble maintainers(oh, sry but I was the person who troubled them most
@mumoshu oops, wanted to reply to your comment
Yes, that use-case is definitely more difficult to address, and I agree with all of the above.
I added my examples to show possible ways to work around the current limitations, and those examples may (or may not) fit your use case (I've seen many users complain about missing features (in general, not just this case) that could be addressed by a small change in their process)
TL;DR; while there are ways to address the problem for some use-cases, the build-time secrets problem is not yet resolved. Work is in progress on BuildKit (https://github.com/moby/buildkit), which provides a lot of low-level features that could be used to power
Yes; #13490 lists concerns/missing things for build-time secrets; feel free to participate there