Declarative containers ignore networking.nameservers, even with privateNetwork=true

[delroth]

Describe the bug

NixOS declarative containers have a "feature" that overwrites the container's /etc/resolv.conf at startup with the one from the host. I have doubts about that implementation in the general case, but in the case of a container configured with privateNetwork = true; it is beyond broken. Not only does it copy a resolv.conf that will in all likelihood not work (because we're in a private network namespace, with different interfaces / addressing), but it also somehow confuses resolvconf and makes networking.nameservers not do anything by default.

There is a workaround: environment.etc."resolv.conf".text = "..."; but that's very counter-intuitive to find, and we shouldn't require that for basic networking configuration.

I'm not entirely sure how to fix this, it seems like there are two issues compounded with each other here. First, I think copying resolv.conf with privateNetwork = true just shouldn't happen, if it ever does something useful I suspect that's accidental. Second, resolvconf should be overwriting that host resolv.conf file with what has been configured in networking.nameservers, but it doesn't do it, and I don't yet understand why. It just decides to not touch the file and copy it verbatim.

Steps To Reproduce

  containers.test = {
    privateNetwork = true;
    config = {
      networking.nameservers = [ "8.8.8.8" ];
    };

    # irrelevant to this bug
    autoStart = true;
    ephemeral = true;
  };

host# nixos-container root-login test
test# cat /etc/resolv.conf
# Generated by resolvconf
search delroth.net
nameserver 127.0.0.53
options edns0 trust-ad
options edns0

Expected behavior

A declarative container can define its own DNS configuration without getting trampled over by a bash script.

Screenshots

n/a

Additional context

The original implementation of the resolv.conf copying dates back to 2014: e620be9

Notify maintainers

Tagging @abbradar because I suspect from git history they might know about resolvconf, but that's a shot in the dark :-)

Metadata

• system: "x86_64-linux"
• host os: Linux 5.15.16-hardened1, NixOS, 22.05 (Quokka)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.7.0pre20220221_caf5172
• channels(delroth): ""
• nixpkgs: /etc/nixpkgs

[aanderse]

@Ma27 does your rewrite of the containers have this same behaviour?

[Ma27]

You can use the ResolvConf= setting from systemd.nspawn(5). With that you can either forward the host's resolv.conf or not. The container-side uses networkd so you can configure your own networkd/resolved setup within the container as well.

The whole point of the change is to support as much of the usptream features as possible.

Btw, I know that the PR requires a bit of work, I'll be on vacation now, after that I'll hope that I'll get to work on that :)

[onny]

@Ma27 Can you please give a more concrete example. I'm not yet able to configure nameservers of the NixOS container :(

[steveej]

i think the culprit is the following:

nixpkgs/nixos/modules/virtualisation/container-config.nix

Line 7 in 50add64

config = mkIf config.boot.isContainer {

nixpkgs/nixos/modules/virtualisation/container-config.nix

Line 14 in 50add64

networking.useHostResolvConf = mkDefault true;

[voidzero]

Perhaps a better default is to use 1.0.0.1 and 1.1.1.1.

[Ultra980]

I might be wrong here, but doesn't

nixpkgs/nixos/modules/virtualisation/container-config.nix

Line 14 in 50add64

networking.useHostResolvConf = mkDefault true;

mean that setting networking.useHostResolvConf = mkForce false; in the container's config fix this?

[onny]

Bug is still present in NixOS 23.11. I described a workaround here on the wiki page https://nixos.wiki/wiki/NixOS_Containers

[voidzero]

Completed how, any comment?

[shlevy]

I think I’m hitting this issue on 23.11, is it fixed on master?

[cmoog]

#307173 Sigh

[onny]

Bug is still present on master