Remote Dockerfiles

[BradleyMcCallionCOT]

Description

I feel like this must have been something that has been requested or asked about before but why is there no option to target a remote Dockerfile. There seems to be options for targeting remote build contexts and even remote docker bake configurations but not for the Dockerfile itself. Is this by design?

My use case is that I want to maintain a centralised Dockerfile across multiple similar microservices. I then want to specify the configuration differences via a docker bake file contained in the microservices themselves. However the one downside to this is that it means developers are restricted from building locally unless they clone down the repo where the centralised dockerfile is maintained and make sure to update it and keep it at a consistent path. It also doesn't allow for versioning like a remote reference would with the #ref:dir syntax

[thaJeztah]

Hmm.. so it's actually already supported, but for some reason it didn't make it into the documentation 🤔
https://docs.docker.com/reference/cli/docker/buildx/build/#file

I wondered if it perhaps got lost in a rewrite, but the old location of the docker build reference docs also didn't mention it

So the short answer is that it's supported; the -f option on docker build / docker buildx build allows specifying a local path, - ( to read from stdin), or a URL.

For example;

This example builds using the dockerfile from https://github.com/docker/cli/blob/v28.0.4/dockerfiles/Dockerfile.shellcheck, but with the local directory as build-context (.); (build fails in this example, because it can't find paths it's looking for);

docker build -f https://raw.githubusercontent.com/docker/cli/refs/tags/v28.0.4/dockerfiles/Dockerfile.shellcheck .
[+] Building 2.8s (10/10) FINISHED                                                                                                     docker:desktop-linux
 => CACHED [internal] load https://raw.githubusercontent.com/docker/cli/refs/tags/v28.0.4/dockerfiles/Dockerfile.shellcheck                            0.0s
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 332B                                                                                                                   0.0s
 => resolve image config for docker-image://docker.io/docker/dockerfile:1                                                                              1.0s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:4c68376a702446fc3c79af22de146a148bc3367e73c25a5803d453b6b3f722fb                        0.0s
 => => resolve docker.io/docker/dockerfile:1@sha256:4c68376a702446fc3c79af22de146a148bc3367e73c25a5803d453b6b3f722fb                                   0.0s
 => [internal] load metadata for docker.io/koalaman/shellcheck-alpine:v0.7.1                                                                           0.6s
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 45B                                                                                                                       0.0s
 => [shellcheck 1/3] FROM docker.io/koalaman/shellcheck-alpine:v0.7.1@sha256:d6147f30864ddb7c9cf983fc277d345bc315e798e309ddf70062b194843ee252          0.0s
 => => resolve docker.io/koalaman/shellcheck-alpine:v0.7.1@sha256:d6147f30864ddb7c9cf983fc277d345bc315e798e309ddf70062b194843ee252                     0.0s
 => [internal] load build context                                                                                                                      0.8s
 => => transferring context: 837.71kB                                                                                                                  0.8s
 => CACHED [shellcheck 2/3] WORKDIR /go/src/github.com/docker/cli                                                                                      0.0s
 => ERROR [shellcheck 3/3] RUN --mount=type=bind,target=.   set -eo pipefail;   find scripts/ contrib/completion/bash -type f | grep -v scripts/winre  0.2s
------
 > [shellcheck 3/3] RUN --mount=type=bind,target=.   set -eo pipefail;   find scripts/ contrib/completion/bash -type f | grep -v scripts/winresources | grep -v '.*.ps1' | xargs shellcheck:
0.140 find: scripts/: No such file or directory
...

The build-context can be specified separately, so instead of the current directory, the example below builds from the v27.5.1 tag from the https://github.com/docker/cli repository;

docker build -f https://raw.githubusercontent.com/docker/cli/refs/tags/v28.0.4/dockerfiles/Dockerfile.shellcheck https://github.com/docker/cli.git#v27.5.1
[+] Building 4.1s (12/12) FINISHED                                                                                                     docker:desktop-linux
 => CACHED [internal] load https://raw.githubusercontent.com/docker/cli/refs/tags/v28.0.4/dockerfiles/Dockerfile.shellcheck                            0.0s
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 332B                                                                                                                   0.0s
 => resolve image config for docker-image://docker.io/docker/dockerfile:1                                                                              2.4s
 => [auth] docker/dockerfile:pull token for registry-1.docker.io                                                                                       0.0s
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:4c68376a702446fc3c79af22de146a148bc3367e73c25a5803d453b6b3f722fb                        0.0s
 => => resolve docker.io/docker/dockerfile:1@sha256:4c68376a702446fc3c79af22de146a148bc3367e73c25a5803d453b6b3f722fb                                   0.0s
 => [internal] load metadata for docker.io/koalaman/shellcheck-alpine:v0.7.1                                                                           0.6s
 => [auth] koalaman/shellcheck-alpine:pull token for registry-1.docker.io                                                                              0.0s
 => [shellcheck 1/3] FROM docker.io/koalaman/shellcheck-alpine:v0.7.1@sha256:d6147f30864ddb7c9cf983fc277d345bc315e798e309ddf70062b194843ee252          0.0s
 => => resolve docker.io/koalaman/shellcheck-alpine:v0.7.1@sha256:d6147f30864ddb7c9cf983fc277d345bc315e798e309ddf70062b194843ee252                     0.0s
 => [internal] load git source https://github.com/docker/cli.git#v27.5.1                                                                               0.8s
 => CACHED [shellcheck 2/3] WORKDIR /go/src/github.com/docker/cli                                                                                      0.0s
 => CACHED [shellcheck 3/3] RUN --mount=type=bind,target=.   set -eo pipefail;   find scripts/ contrib/completion/bash -type f | grep -v scripts/winr  0.0s
 => exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                0.0s
 => => exporting manifest sha256:cd17985409ba8077fee662896b872404e15e182df37fb9335e5c4f3f28030a27                                                      0.0s
 => => exporting config sha256:2847d258efbff61bf87a194862f8d447512a421d2c15c0c1f9c6f3a50d173b5f                                                        0.0s
 => => exporting attestation manifest sha256:eb7abc5cdeafea1e58e78085cd4ade3cb66333e9dcbddb6f67fbf767842d5f77                                          0.0s
 => => exporting manifest list sha256:cea7e58379ab3529b3899b5290ff22ecd93495ba8b3725c808b798b7dee0b0f8                                                 0.0s
 => => naming to moby-dangling@sha256:cea7e58379ab3529b3899b5290ff22ecd93495ba8b3725c808b798b7dee0b0f8                                                 0.0s
 => => unpacking to moby-dangling@sha256:cea7e58379ab3529b3899b5290ff22ecd93495ba8b3725c808b798b7dee0b0f8                                              0.0s

[BradleyMcCallionCOT]

@thaJeztah Thanks for your reply. Okay good to know. Does this support private git repositories the same way you can with remote bake definitions? And does it support the #ref:dir syntax? The only way I can get around this at the moment is by instead defining the dockerfile inline within a remote docker bake definition then chaining docker bake definitions overriding this remote definition with a local definition that provides the relevant arguments. What would also work is being able to load a remote Dockerfile through a remote Bake file definition. For example suppose the remote repo has the structure

-- docker-bake.hcl
-- Dockerfile

When targeting this remote bake file via docker buildx bake git@ssh..., there doesn't seem to be a way to load the Dockerfile sitting next to the docker bake definition. The target.dockerfile argument seems to try resolve paths locally. It seems the only workaround to this is as I put above instead defining the Dockerfile inline, but this loses a lot of the linting capabilities of IDEs and is generally a bit messier to edit.

[BradleyMcCallionCOT]

On one related but separate question. Is there a way to inherit a docker bake definition in a file rather than a command? I.e. rather than running docker buildx bake -f docker-bake.hcl -f cwd://docker-bake.override.hcl git@ssh....#ref:dir we can pass the argument docker-bake.hcl and git@ssh....#ref:dir in the docker-bake.override.hcl file somewhere? Then simply run docker build bake locally?

[thaJeztah]

Does this support private git repositories the same way you can with remote bake definitions?

That's a good question; not sure if it does! I saw your ticket, and recalled that remote Dockerfiles were supported (but from when it was implemented as part of the classic (pre-buildkit) builder). I'm not sure if BuildKit changed that to take advantage of the more advanced mechanism it supoorts. Possibly @tonistiigi or @crazy-max can fill in the blanks there.

On one related but separate question. Is there a way to inherit a docker bake definition in a file rather than a command?

I'll have to defer that question as well (at least, I'd have to dig in to that myself as well!). I know there's parts where (by design) some things must be passed when invoking the build commands (e.g. a Dockerfile cannot dictate the image:tag the image should be built as). Bake-files are definitely different in that respect, as they are somewhat similar to a Makefile, so it's more along the line of expectations for those to specify such things, but I'm not sure if it currently supports that.

(interested to know though!)

[BradleyMcCallionCOT]

@thaJeztah Thanks again for your prompt reply! Hopefully @tonistiigi or @crazy-max can fill in the blanks. If it doesn't then there's my feature requests!

[tonistiigi]

Does this support private git repositories the same way you can with remote bake definitions? And does it support the #ref:dir syntax?

There is no syntax for git URL that points to a specific file. You can define context with a Git URL (or directory from that repo) and then dockerfile relative to that context.

Is there a way to inherit a docker bake definition in a file rather than a command?

No, there is no import-like functionality atm in hcl. I do think compose files have some capabilities like this but don't know if there are some limitations.

[BradleyMcCallionCOT]

@tonistiigi Thanks okay I seem to have sorted this by defining the main build context as the remote repo where the centralised Dockerfile is stored and then using a relative reference from their for the Dockerfile. I then defined another context using the target.contexts attribute called target-repo which pointed to cwd:// (i.e. where the docker build/bake command is being run from). In the centralised Dockerfile itself I just make sure all COPY commands use --from target-repo.

[thaJeztah]

If you (or someone else) is interested in contributing to update the documentation for mentioning what's currently supported (dockerfile from remote URL); the source for the documentation on the -f / --file flag for docker buildx build can be found here; https://github.com/docker/buildx/blob/v0.23.0/docs/reference/buildx_build.md#file

And for the bake-file target.dockerfile; https://github.com/docker/buildx/blob/v0.23.0/docs/bake-reference.md#targetdockerfile

(but at a quick glance, that is referencing the --file flag, so probably don't needs updating)

[polarathene]

Potentially related is remote builds lack the ability for a remote .dockerignore (Dockerfile specific overrides or the main .dockerignore all need to be supplied locally when running docker build), regardless of building the implicit Dockerfile from remote build context, or when using a remote --file.

From the linked docs page:

When you run a build command, the build client looks for a file named .dockerignore in the root directory of the context.
If this file exists, the files and directories that match patterns in the files are removed from the build context before it's sent to the builder.

Seems a bit odd that you cannot supply an override for that (or use the one co-located with a remote build context, such as a repo). Thus you may copy over a fair bit more than intended.

---

UPDATE: Nevermind, there's no such support for applying .dockerignore to remote build context. Even if that is available locally the COPY from the Dockerfile at the remote git repo adds everything, disregarding any .dockerignore 🤷‍♂ (that correction is communicated here, just not yet reflected on official docs)