Clean up localhost resolv logic and add IPv6 support to regexp #10005

estesp · 2015-01-09T17:21:15Z

Addresses #5811

This cleans up an error in the logic which removes localhost resolvers
from the host resolv.conf at container creation start time. Specifically
when the determination is made if any nameservers are left after
removing localhost resolvers, it was using a string match on the word
"nameserver", which could have been anywhere (including commented out)
leading to incorrect situations where no nameservers were left but the
default ones were not added.

This also adds some complexity to the regular expressions for finding
nameservers in general, as well as matching on localhost resolvers due
to the recent addition of IPv6 support. Because of IPv6 support now
available in the Docker daemon, the resolvconf code is now aware of
IPv6 enable/disable state and uses that for both filter/cleaning of
nameservers as well as adding default Google DNS (IPv4 only vs. IPv4
and IPv6 if IPv6 enabled). For all these changes, tests have been
added/strengthened to test these additional capabilities.

Docker-DCO-1.1-Signed-off-by: Phil Estes estesp@linux.vnet.ibm.com (github: estesp)

jessfraz · 2015-01-12T17:26:21Z

will this fix #5811

estesp · 2015-01-12T18:10:16Z

@jfrazelle interesting.. it doesn't in the current form, but given what I've added, it would be relatively simple to add a check for "if IPv6" enabled, and clean out all IPv6 nameservers rather than just localhost if IPv6 is currently not enabled for the container. If that makes sense I can update the PR with that capability.

jessfraz · 2015-01-12T18:14:41Z

ya that would be awesome! this is the problem we alsways have w the drone
server and it is such a PITA

On Mon, Jan 12, 2015 at 10:10 AM, Phil Estes notifications@github.com
wrote:

@jfrazelle https://github.com/jfrazelle interesting.. it doesn't in the
current form, but given what I've added, it would be relatively simple to
add a check for "if IPv6" enabled, and clean out all IPv6 nameservers
rather than just localhost if IPv6 is currently not enabled for the
container. If that makes sense I can update the PR with that capability.

—
Reply to this email directly or view it on GitHub
#10005 (comment).

estesp · 2015-01-12T21:42:36Z

@jfrazelle The updated PR will now fix #5811: if daemon.config.IPv6Enable is false, the "localhost resolver" filtering function is now generalized to clean the host resolv.conf suitably for the container, including for IPv4 vs IPv4+6 issues. I also added the default Google IPv6 DNS servers and they will be applied if IPv6 is enabled in the daemon and no valid servers are left in resolv.conf after cleaning up localhost resolvers. All the tests are unit tests at this point as the IPv6 support at runtime would depend on either running a test daemon with IPv6 enabled with integration-cli or adding more tests which run a new daemon with private config to test IPv6 on vs. off scenarios.

MalteJ · 2015-01-12T22:35:22Z

pkg/networkfs/resolvconf/resolvconf.go

+	defaultIPv6Dns = []string{"2001:4860:4860::8888", "2001:4860:4860::8844"}
+	ipv4NumBlock   = `(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)`
+	ipv4Address    = `(` + ipv4NumBlock + `\.){3}` + ipv4NumBlock
+	ipv6Address    = `([0-9A-Fa-f]{0,4}:){2,7}([0-9A-Fa-f]{1,4})`


I think it is possible to skip the last digit if it's a 0. e.g. 2001:db8::
I would use

ipv6Address = `([0-9A-Fa-f]{0,4}:){2,7}([0-9A-Fa-f]{0,4})`

Wouldn't this really need to be something insane like in
http://stackoverflow.com/a/17871737/433558 ? (so that we match all
permutations of possible addresses, like IPv4-Embedded IPv6 Addresses)

I think it will match a superset of IPv6 addresses.
e.g. invalid addresses like fe80:::1 as well as transition addresses like ::ffff:0:a:b:c:d

Besides, link-local IPv6 addresses like fe80::a:b:c:d%eth0 that work on the host will not work from within the container.
But I think the person who's using link-local addresses for DNS should know what he's doing (wrong).

EDIT
Oh, I didn't know those IPv4-Embedded IPv6 Addresses.
Well. Yeah. Maybe there will be a Pull-Request for this - probably not 😄

@MalteJ @tianon given the complexity of an exact regex, I chose something that was a bit less aggressive given what we are coming from (at least from IPv4 nameserver perspective) was not even refusing invalid IPv4.. and given this particular code is not trying to explicitly validate, but rather match, it seemed close enough from all my investigation. I did shorten it to not handle the IPv4-embedded, but that can be added back in to the regex if we feel that will be a valid use case. Since we have dual-stack, not sure why embedded IPv4 would be that useful.

I agree, @estesp
But I would still use {0,4} at the end.

Thanks for this PR! 😃

jessfraz · 2015-01-12T22:59:56Z

cool thanks!

estesp · 2015-01-13T18:09:54Z

@tianon @MalteJ thanks for the review and comments--I've updated the IPv6 regex with the suggestion, and made a comment about the choice of simplicity over exactness so others who have questions can easily discern the reason we used this regex.

jessfraz · 2015-01-13T18:28:24Z

I appreciate all the code comments LGTM

LK4D4 · 2015-01-13T18:29:45Z

pkg/networkfs/resolvconf/resolvconf.go

+	// if the resulting resolvConf has no more nameservers defined, use defaultDns
+	if len(GetNameservers(cleanedResolvConf)) == 0 {
+		log.Infof("No non-localhost DNS nameservers are left in resolv.conf. Using default external servers : %v", defaultIPv4Dns)
+		cleanedResolvConf = append(cleanedResolvConf, []byte("\nnameserver "+strings.Join(defaultIPv4Dns, "\nnameserver "))...)


this Join looks superweird :) Let's make default nameservers as just strings:

nameserver 8.8.8.8 nameserver 8.8.8.4

Good point--that was weird; fixed in latest amend to commit!

jessfraz · 2015-01-19T21:58:37Z

ping @LK4D4

LK4D4 · 2015-01-19T23:45:38Z

pkg/networkfs/resolvconf/resolvconf.go

+	// default DNS servers for IPv4 and (optionally) IPv6
+	if len(GetNameservers(cleanedResolvConf)) == 0 {
+		log.Infof("No non-localhost DNS nameservers are left in resolv.conf. Using default external servers : %v", defaultIPv4Dns)
+		cleanedResolvConf = append(cleanedResolvConf, []byte("\n"+strings.Join(defaultIPv4Dns, "\n"))...)


I think it's cleaner to combine array of needed dns, like dns := append(defaultIPv4Dns, defaultIPv6Dns) and then use all your Join magic.

Addresses moby#5811 This cleans up an error in the logic which removes localhost resolvers from the host resolv.conf at container creation start time. Specifically when the determination is made if any nameservers are left after removing localhost resolvers, it was using a string match on the word "nameserver", which could have been anywhere (including commented out) leading to incorrect situations where no nameservers were left but the default ones were not added. This also adds some complexity to the regular expressions for finding nameservers in general, as well as matching on localhost resolvers due to the recent addition of IPv6 support. Because of IPv6 support now available in the Docker daemon, the resolvconf code is now aware of IPv6 enable/disable state and uses that for both filter/cleaning of nameservers as well as adding default Google DNS (IPv4 only vs. IPv4 and IPv6 if IPv6 enabled). For all these changes, tests have been added/strengthened to test these additional capabilities. Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)

estesp · 2015-01-20T00:37:46Z

@LK4D4 updated--PTAL!

LK4D4 · 2015-01-20T18:30:03Z

LGTM

Clean up localhost resolv logic and add IPv6 support to regexp

estesp force-pushed the fix-localhost-nameserver-cleanup branch from 0335048 to 2708f8e Compare January 12, 2015 21:36

MalteJ reviewed Jan 12, 2015
View reviewed changes

estesp force-pushed the fix-localhost-nameserver-cleanup branch from 2708f8e to db4b472 Compare January 13, 2015 18:08

LK4D4 reviewed Jan 13, 2015
View reviewed changes

estesp force-pushed the fix-localhost-nameserver-cleanup branch from db4b472 to c74479c Compare January 13, 2015 20:07

crosbymichael added this to the 1.5.0 milestone Jan 14, 2015

estesp mentioned this pull request Jan 19, 2015

Fix up nameserver localhost detection #10126

Closed

LK4D4 reviewed Jan 19, 2015
View reviewed changes

estesp force-pushed the fix-localhost-nameserver-cleanup branch from c74479c to 93d51e5 Compare January 20, 2015 00:36

LK4D4 added a commit that referenced this pull request Jan 20, 2015

Merge pull request #10005 from estesp/fix-localhost-nameserver-cleanup

e9d3e23

Clean up localhost resolv logic and add IPv6 support to regexp

LK4D4 merged commit e9d3e23 into moby:master Jan 20, 2015

SvenDowideit mentioned this pull request Jan 27, 2015

Post 1.4.1 docs update 2 #9739

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up localhost resolv logic and add IPv6 support to regexp #10005

Clean up localhost resolv logic and add IPv6 support to regexp #10005

estesp commented Jan 9, 2015

jessfraz commented Jan 12, 2015

estesp commented Jan 12, 2015

jessfraz commented Jan 12, 2015

estesp commented Jan 12, 2015

MalteJ Jan 12, 2015

tianon Jan 12, 2015

MalteJ Jan 12, 2015

MalteJ Jan 12, 2015

estesp Jan 12, 2015

MalteJ Jan 13, 2015

jessfraz commented Jan 12, 2015

estesp commented Jan 13, 2015

jessfraz commented Jan 13, 2015

LK4D4 Jan 13, 2015

estesp Jan 13, 2015

jessfraz commented Jan 19, 2015

LK4D4 Jan 19, 2015

estesp commented Jan 20, 2015

LK4D4 commented Jan 20, 2015

Clean up localhost resolv logic and add IPv6 support to regexp #10005

Clean up localhost resolv logic and add IPv6 support to regexp #10005

Conversation

estesp commented Jan 9, 2015

jessfraz commented Jan 12, 2015

estesp commented Jan 12, 2015

jessfraz commented Jan 12, 2015

estesp commented Jan 12, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessfraz commented Jan 12, 2015

estesp commented Jan 13, 2015

jessfraz commented Jan 13, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessfraz commented Jan 19, 2015

Choose a reason for hiding this comment

estesp commented Jan 20, 2015

LK4D4 commented Jan 20, 2015