*: DNS functionality is not complete

[steveej]

• Document all the cases a user has in order to (not) have the resolv.conf for the Pod's applications (over)written
• Support DNS results from CNI
• Populate resolv.conf automatically without user input
• Improve tests
• Support setting the Pod's domain name (Set domain name in rkt pods #2042)

[jonboulle]

@steveej please do this or find another owner

[alban]

@steveej: can you help to prioritize each task?

[tmrts]

@steveej can we do something about this?

[Andrei-Pozolotin]

any chance to increase priority for:

• Support DNS results from CNI ?

[squeed]

As for populating /etc/resolv.conf without user input, this would be a backwards-compatability breaking change. I propose adding an option (probably --host-dns or --dns=host) which will read-only bind-mount the host's /etc/resolv.conf.

[squeed]

I am currently working on CNI-DNS. A question: what do we do if multiple network plugins return DNS information? Or, if stage0 was supplied --dns?

[Andrei-Pozolotin]

re: multiple network plugins return DNS information?

1. make it configurable
2. support different behaviors

• --dns=cni:use-first
• --dns=cni:use-last
• --dns=cni:merge-all

[monder]

I thought that cni order is not guaranteed. And there are some edge cases with default and host networks. I suggest csv as with --net itself.

--dns=host
--dns=host,cni:flannel
--dns=cni:flannel,host

[Andrei-Pozolotin]

@monder +1

[squeed]

For CNI order is not guaranteed, but rkt at least claims to enforce lexicographic ordering :-). In any case, we should accept CNI DNS results without requiring them to be explicitly enabled?

[Andrei-Pozolotin]

re: explicitly enabled?

1. if there is a place for global defaults, like /etc/rkt/net.d(?) - control from there
2. if not, then must be explicitly enabled

[squeed]

After talking with @steveej, here is my proposal for how DNS should work.

In order of precedence:

1. the --dns flags at runtime. I intend to add two new special values for the --dns flag:

   • --dns=host: bind-mount the host's /etc/resolv.conf in to the pod
   • --dns=none: Do not manage DNS at all, letting each application manage it. (Skip CNI)

   Passing multiple --dns{-opt|search} flags when one value is magic is an error.

2. The first non-empty DNS value from a network configuration file or CNI plugin

3. The individual application's /etc/resolv.conf

The multiple interpretations of the --dns flag are somewhat awkward. I think, however, that this is preferable to adding a new flag. Ideally there would have been --dns and --dns-server but I'd like to keep the API consistent.

[monder]

Passing multiple --dns{-opt|search} flags when one value is magic is an error.

What do you mean by 'magic'?

The first non-empty DNS value from a network configuration file or CNI plugin

So basically if I just run rkt run --net=default,net1, the resolv.conf will come from net1 because the default has no configuration, right?

And does that mean that if I don't specify any third-party CNI, there will be no dns, and I should always enable it via --dns=host explicitly?

[squeed]

@monder

Passing multiple --dns{-opt|search} flags when one value is magic is an error.
What do you mean by 'magic'?

The standard meaning of --dns=1.2.3.4 is to create an /etc/resolv.conf with nameserver 1.2.3.4. I'm proposing overloading this flag with some non-IP options. That's what I mean by 'magic' values.

So basically if I just run rkt run --net=default,net1, the resolv.conf will come from net1 because the default has no configuration, right?

Correct, but be careful. Rkt's network ordering always comes from the lexicographic ordering of the network configuration files. The order specified on the command line has no effect. The default network always has the name 99-default.conf.

And does that mean that if I don't specify any third-party CNI, there will be no dns, and I should always enable it via --dns=host explicitly?

Correct. This is basically the state we are in now - /etc/resolv.conf is never implicitly created.

[squeed]

As a followup, it seems like rkt-resolv.conf generation / management will have to be moved to the stage1s. We shouldn't be doing bind-mounts in stage0, especially with a view to cross-platform.

[squeed]

There is a potential backwards-compatability-breaking change needed for setting the DNS domain name. In the case where a CNI plugin supplies a domain name AND an application in the pod provides an /etc/hosts, we will have to overwrite it with our own.

By default, I think this is OK - /etc/hosts in most images is just the default file included with each distribution. We shouldn't merge the files, because pods should have consistent hosts files.

So, is anyone concerned with this case overwriting the app's /etc/hosts?