New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: Support the /etc/resolver DNS resolution configuration hierarchy on OS X #12524

Open
Rotonen opened this Issue Sep 6, 2015 · 18 comments

Comments

Projects
None yet
@Rotonen

Rotonen commented Sep 6, 2015

https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man5/resolver.5.html

OS X allows you to add TLD specific resolver configurations. Quite popular ones are /etc/resolver/vm for local virtual machines and /etc/resolver/dev for local development purposes.

https://golang.org/src/net/dnsclient_unix.go#L231

Go seems to be hardcoded to only take /etc/resolv.conf into account on Unix platforms.

@nodirt

This comment has been minimized.

Member

nodirt commented Sep 6, 2015

I don't think Go-native DNS resolving mechanism is used on Mac.
https://golang.org/src/net/dnsclient_unix.go#L231 is not executed if I run

addrs, err := net.LookupHost("google.com")

on my Mac.

If I enable debugging (GODEBUG=netdns=2 go run test.go), the following is printed:

go package net: using cgo DNS resolver
go package net: hostLookupOrder(google.com) = cgo

which means that OS-native DNS resolving is used.

Can you supply an exact configuration file, Go code, actual and expected output?

@titanous

This comment has been minimized.

Member

titanous commented Sep 6, 2015

@nodirt This is for a binary with cgo off.

@davecheney

This comment has been minimized.

Contributor

davecheney commented Sep 6, 2015

If cgo is disabled then the pure go DNS resolver will be used. If you want
to use the Mac DNS resolver, plese build with cgo.

On Mon, 7 Sep 2015 07:47 Jonathan Rudenberg notifications@github.com
wrote:

@nodirt https://github.com/nodirt This is for a binary with cgo off.


Reply to this email directly or view it on GitHub
#12524 (comment).

@nodirt

This comment has been minimized.

Member

nodirt commented Sep 6, 2015

Shouldn't be a problem since this is needed only on a dev machine.

On Sun, Sep 6, 2015 at 4:06 PM Dave Cheney notifications@github.com wrote:

If cgo is disabled then the pure go DNS resolver will be used. If you want
to use the Mac DNS resolver, plese build with cgo.

On Mon, 7 Sep 2015 07:47 Jonathan Rudenberg notifications@github.com
wrote:

@nodirt https://github.com/nodirt This is for a binary with cgo off.


Reply to this email directly or view it on GitHub
#12524 (comment).


Reply to this email directly or view it on GitHub
#12524 (comment).

@titanous

This comment has been minimized.

Member

titanous commented Sep 6, 2015

In this specific case, @Rotonen was using the Flynn binary that we distribute as a compiled artifact, it is compiled without cgo to ease cross-compilation. Just because the user is a developer doesn't mean that they are a Go developer or want to compile the binary for themselves. The only question here is if this feature is out of scope for the pure-Go resolver.

@minux

This comment has been minimized.

Member

minux commented Sep 7, 2015

@ianlancetaylor ianlancetaylor changed the title from Support the /etc/resolver DNS resolution configuration hierarchy on OS X to net: Support the /etc/resolver DNS resolution configuration hierarchy on OS X Sep 8, 2015

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Sep 8, 2015

I don't see anything wrong with supporting the OS X /etc/resolver directory. That said, my understanding is that the Go DNS resolver does not work well on most OS X machines. That is why it is disabled by default.

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Sep 8, 2015

@mterron

This comment has been minimized.

mterron commented Jun 1, 2016

This would be great in all platforms anyway. Is there any disadvantage from supporting this behaviour? It seems that it'd neatly resolve the need to install and configure dnsmasq to provide the simple function of having different resolvers for different TLDs.

@jason-riddle

This comment has been minimized.

jason-riddle commented Feb 16, 2017

i know this issue is quite old but has there been any traction on this?

@pantocrator27

This comment has been minimized.

pantocrator27 commented Jun 23, 2017

any resolution?

@bradfitz

This comment has been minimized.

Member

bradfitz commented Jun 23, 2017

Any updates would be posted here. No updates have been posted here.

@bitglue

This comment has been minimized.

bitglue commented Apr 20, 2018

See resolver(5). Just reading the files out of /etc/resolver/* will miss out on other mechanisms for configuring the same thing, for example configuration profiles or IKE attributes.

@flyinprogrammer

This comment has been minimized.

flyinprogrammer commented Sep 25, 2018

Just stumbled upon this today while attempting to use coredns as a dns proxy for local development. It's a real bummer to discover how naive our support for os x is.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Sep 25, 2018

We've generally assumed people use cgo on Darwin, so this bug has never been a priority.

I do admit that practically means that Darwin binaries need to be built on Darwin, which is difficult for people wanting to cross-compile for a dozen platforms as part of their release process.

Perhaps on Darwin without cgo we could just shell out to a program to do DNS resolution (e.g. host, dig, nslookup?). At least nslookup has an interactive mode that would permit re-using a child process for multiple lookups, if that proves necessary for performance.

@bitglue

This comment has been minimized.

bitglue commented Sep 26, 2018

I think reality is most command-line utilities will compile for two platforms: Linux and OS X, and the OS X build will always have cgo disabled. Some subset of the OS X users are using VPN, expect .local names to resolve, or have some other situation where hostname resolution is more than "just query this one DNS server always". Some subset of those users will actually open an issue with the tool, and of those even a smaller subset identify go as the problem and raise an issue here.

So I think you underestimate the impact of the problem.

Shelling out to nslookup will not fix it. The problem is "doing a DNS query" is not the same thing as "resolving a hostname". Resolving a hostname involves more, such as:

  • /etc/hosts
  • RFC6762 .local names
  • Other hostname resolution protocols, such as NIS or LDAP, if configured
  • Honoring the domain search path, if configured
  • If DNS is to be used, determining which server to use.

Tools like host, nslookup, and dig do DNS queries by design, not resolve hostnames. This is equally true on Linux as well as OS X. Unfortunately somehow OS X has acquired some lore about having "two DNS systems", which is simply false. Or at least it was false, until go command-line utilities gained popularity.

If you do want to shell out to a command to perform host resolution, the correct command on OS X is dscacheutil -q host -a name $hostname. This is analogous to getent hosts $hostname on Linux.

Another path is to make the go resolver's behavior more consistent with the OS X system resolver. This begins with obtaining resolver configuration from SystemConfiguration.framework or scutil --dns, not /etc/resolv.conf.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Sep 26, 2018

dscacheutil sounds good. I was thinking of lookupd when I wrote the comment above but my local machine didn't have lookupd so I omitted it. Now I see that dscacheutil replaced lookupd.

I don't think we want to get into the business of reimplementing Darwin's name resolution.

@randall77, since you're having fun with macOS lately, any thoughts here? Could we have non-cgo binaries still call into the macOS name resolution code somehow with some assembly/linker goo?

@rsc

This comment has been minimized.

Contributor

rsc commented Sep 26, 2018

Let's see if we can use the libSystem bindings directly even when cgo is ostensibly disabled.

@Rotonen

This comment has been minimized.

Rotonen commented Sep 26, 2018

expect .local names to resolve

I actually expect .local names to resolve on all platforms per mDNS anyway, if the target responds to the broadcast appropriately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment