Support different DNS resolver if host network is used #2404
Comments
tonistiigi
changed the title
|
I am also having issues with --network=host and buildkit. Here is a description of my setup: My setupLinux machine, connected to company VPN from home. my my Using the simple test Dockerfile: Without Buildkit
This is as expected (it's my actual With Buildkit
And I can't access any of my company hostnames, as the dns resolving fails. |
|
I'm encountering the same regression with BuiltKit being enabled/installed by default in recent Docker versions. Our image builders do not have direct network access. Instead they have a properly configured Unbound installed locally: On the host: With the legacy builder this worked fine with With BuiltKit it pulls Google Public DNS from somewhere and that is not reachable / fails to resolve the hostname of our internal HTTP proxy: I've attempted to configure DNS using I've also attempted to configure DNS in dockers daemon.json, but that is ignored as well. |
Okay that works, but doesn't work if I use 127.0.0.1 in there. I might be able to work around that by using a different IP address pointing to the local machine that is not filtered. It's still a regression and highly unexpected, though. |
|
I think this issue should be marked as a bug and moby/libnetwork#2385 should probably be reverted. Systemd is already symlinking the proper resolver config to Furthermore, For reference, here's the man page Just as a side note. Using the systemd default also solves the problem that buildkit is actually not updating on resolver config changes. I realized that when I tried to redirect I hope this is some useful information ... thx |
|
Now that Buildx has become the default with Docker and the behavior is a breaking change I would also promote this issue to a bug. |
|
I am also experiencing this issue on Arch Linux where my system can correctly resolve a hostname with a local DNS server but because This functionality was working until quite recently and I don't understand why this change would be made? If it's not a bug it is definitely a regression. |
Was "quite recently" since 2023? (v23 of Docker would make sense) If it was with a recent version of buildkit you could try using an earlier release. You probably want to use You can also try setting the DNS to use in |
"Quite recently" as in a couple of weeks ago. We recently transitioned our local registry to HTTPS and we removed all of the insecure registry stuff from our buildx instance and the resolution of domain names using our local DNS server worked fine. I'm not familiar with the extent of what insecure entitlements and
When I sorted out the insecure registries to allow buildx to access our old HTTP registry I was told that buildx doesn't really pay much attention to |
There's been no new releases of Buildx since July: https://github.com/docker/buildx/releases/tag/v0.11.2
So to clarify, by "quite recently" you're saying it was working fine in August? There is two point releases with small changes for BuildKit, with Are you sure it's an issue with BuildKit, or something that changed with Buildx? What version was working before? Buildx can create with older releases for Likewise, with Maybe another change involved elsewhere is contributing to the problem? 🤷♂️
No clue, I'm just a user like yourself. Pretty simple to verify by giving a try though? Only potential inconvenience is that the DNS change isn't scoped to your buildx builds (or specific builds), so having an arg option to provide |
|
My system package for buildx updated from 0.11.0 to 0.11.2 on 2023-08-10. I can't remember the specific date that I last tested the HTTPS registry, but its possible it was around then. I'm not specifically saying that it is a buildx/docker issue, but I am seeing the exact same symptoms that are described earlier in this issue. The Having a look at |
|
TL;DR: I don't see anything in release notes that suggests the breakage you experienced was introduced recently.
That's ok, it'd be good to confirm if rolling back to buildx Updates and related dependencies changes
Thanks! So buildx 0.11.0 (June 2023) was using an 0.11.0 RC version of buildkit (from Dec 2022), and buildx 0.11.2 (July 2023) is using buildkit 0.12.1 (noted as a dev release though), with proper 0.12.1 release in Aug 2023. BuildKit has had quite a bit of changes since then, but nothing stood out as related in the release notes. While buildx 0.11.1 mentions fixing a networking bug with We can also observe between the buildx versions that Docker / The release notes might not be highlighting these kinds of changes, for example there's plenty of DNS activity on Identifying the causeYou'd have to look deeper if it really was a change from these dependencies that changed some behaviour. Obviously it'd help to verify that rolling back your version of buildx does resolve the issue, or if it still occurs then something else has actually introduced it (making the much older history discussed in this issue relevant). I'm not saying that this isn't a bug, as an earlier comment above draws attention to why it is, but they reference reverting a PR from 2019. That bug wouldn't be newly introduced by buildx in the past few weeks. Update:
Getting it resolved upstreamI've been pushing to get a change through with the systemd It took quite a while to get that change approved and merged into The maintainers get a heaps of notifications due to all the activity they're subscribed to, if anyone is interested in seeing a fix my advice is to:
Update: Given my follow-up comment, it seems at least for @Bidski who I assume is using |
|
This is a bit messy and repetitive, sorry about that. I've split it off as a separate part of my prior comment. Spent too much time connecting it all together, only to get many of the same answers relayed to me just as I was finishing up my response 😆 Follow-up UpdateI didn't read through prior comments which are a bit more informative 😅 Original author from Oct 2021 of this issue was specifically pointing out that using BuildKit with The 2020 PR seems to have addressed the concern, and notes the following behaviour:
v23 of Docker enabled BuildKit by default IIRC.
Unrelated to running with host mode networking (possible insight): If it's due to
Update: For
Possible relevant group of issues and stale PR (
Related Oct 2022 issue with registry by @Bidski : docker/buildx#1370 (comment)
So BuildKit does not implement the same
|
|
#2404 (comment) sums it up nicely, thanks, @Ka0o0. Also worth pointing out is that |
|
Has this been resolved in #4524? UPDATE: The reason I'm asking is that I still see this issue, and it has become very unpleasant. |
|
I see that this was fixed and released in https://github.com/moby/buildkit/releases/tag/v0.16.0 But when I test it using v0.16.0, I am still able to repro the issue This is a GitHub Actions workflow to repro the issue The nameserver should have been |
Hi,
I have a host system with systemd-resolved configured. With moby/moby#41022 a more thorough decision concept - on whether
/etc/resolv.confor/run/systemd/resolve/resolv.confshould be selected - was implemented. In short, when I activate the host networking mode during build, I would expect to have/etc/resolv.confcopied from my host into the container. The selection works fine for me for all docker commands, like run and build. I've noticed that when I activate buildkit, however, the behavior is different. It seems like/run/systemd/resolve/resolv.confis used always, regardless of whether the host network option is selected via--network host.Example
Dockerfile
Without Buildkit
docker build --no-cache .docker build --no-cache --network host .With Buildkit
DOCKER_BUILDKIT=1 docker build --no-cache --progress plain .I expect to have
/run/systemd/resolve/resolv.confcopied, as I use the default networking mode (as above).DOCKER_BUILDKIT=1 docker build --no-cache --progress plain --network host .I expect to have
/etc/resolv.conffrom my host copied, as I use the default networking mode host.But instead, the same file as above was copied to the image.
I would expect the behavior to be same after enabling buildkit.
The text was updated successfully, but these errors were encountered: