Pure go dns resolver return directly if found entry in /etc/hosts

[YanzhaoLi]

What version of Go are you using (go version)?

$ go version
1.14.2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go/"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/var/vcap/store/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/var/vcap/store/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build914493944=/tmp/go-build -gno-record-gcc-switches"

What did you do?

dns settings

• there's one dns entry in the /etc/hosts

10.0.16.6 8891a4b1-aec2-4e5e-8741-d3ad88379120.pks-db.scratch-gorilla-management-subnet.pivotal-container-service-9e749684f14fd37a4dc4.bosh

• dns server has two dns entries for this IP

6.8.0.10.in-addr.arpa.  0       IN      PTR     8d71b446-4102-446a-a479-dec89412ec07.master.scratch-gorilla-services-subnet.service-instance-941a792e-f7cc-423a-9171-d63c792f5e8d.bosh.
6.8.0.10.in-addr.arpa.  0       IN      PTR     master-2.etcd.cfcr.internal.

dig -x 10.0.8.6 return 2 domains

; <<>> DiG 9.10.3-P4-Ubuntu <<>> -x 10.0.8.6
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24749
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;6.8.0.10.in-addr.arpa.         IN      PTR
;; ANSWER SECTION:
6.8.0.10.in-addr.arpa.  0       IN      PTR     8d71b446-4102-446a-a479-dec89412ec07.master.scratch-gorilla-services-subnet.service-instance-941a792e-f7cc-423a-9171-d63c792f5e8d.bosh.
6.8.0.10.in-addr.arpa.  0       IN      PTR     master-2.etcd.cfcr.internal.
;; Query time: 0 msec
;; SERVER: 169.254.0.2#53(169.254.0.2)
;; WHEN: Tue Nov 24 06:08:16 UTC 2020
;; MSG SIZE  rcvd: 270

The pure go dnsResolver only return one if there's one entry in /etc/hosts

lnames, lerr := net.DefaultResolver.LookupAddr(ctx, 10.0.8.6)
10.0.8.6 : [8d71b446-4102-446a-a479-dec89412ec07.master.scratch-gorilla-services-subnet.service-instance-941a792e-f7cc-423a-9171-d63c792f5e8d.bosh.]

I checked the golang code here:

go/src/net/dnsclient_unix.go

Line 741 in 750b372

names := lookupStaticAddr(addr)

that the pure go resolver hardcoded check /etc/hosts/ first, and if found, return.

func (r *Resolver) goLookupPTR(ctx context.Context, addr string) ([]string, error) {
	names := lookupStaticAddr(addr)
	if len(names) > 0 {
		return names, nil
	}

What did you expect to see?

net.DefaultResolver.LookupAddr(ctx, 10.0.8.6) return two entries

What did you see instead?

net.DefaultResolver.LookupAddr(ctx, 10.0.8.6) return only one entry

[antong]

Have you configured your system to not use /etc/hosts, i.e., edited /etc/nsswitch.conf to not have "files" before "dns" on a line starting with "hosts:" ?

I believe the intended behavior is for the pure Go resolver (GODEBUG=netdns=go) to mimic the behavior of the system resolver (GODEBUG=netdns=cgo) as closely as reasonable. If nsswitch.conf is configured to use /etc/hosts (e.g., hosts: files dns), then I think what you describe actually is intended behavior: Go sees that you have configured your system to resolve 10.0.8.6 to a single name by having a single entry in /etc/hosts. If you add the second name for 10.0.8.6 to /etc/hosts, then the Go resolver will return both. If you remove the entry from /etc/hosts altogether, then the Go resolver will use DNS and again return both.

I tested, and I can confirm on my end that if I do configure nsswitch to not use /etc/hosts, then the pure Go resolver still does use it for reverse lookups (LookupAddr()), and this sounds like a bug. Forward lookups like LookupHost() works as expected.

The way I've understood things: The "pure Go resolver" is not a "pure DNS client". I hope the docs don't make it sound like that is the case. The Go resolver attempts to mimic and implement in Go the behavior of the C library or system resolver that other programs use. This means that it reads /etc/nsswitch.conf, /etc/resolv.conf and /etc/hosts and acts on these. The intention is that Go programs resolve names and addresses the same way as other programs like C and Python programs. The system administrator can configure name resolution for the whole system by editing e.g., nsswitch.conf or /etc/hosts, and all programs will use this. The admin doesn't need to separately configure Go programs and other programs.

dig is not necessarily a good example, because it does not use a resolver but is explicitly a DNS query tool. If you use curl, ssh, ping, you will find that they too use nsswitch, /etc/hosts etc., as they should. If a DNS client for Go is what you are looking for, then https://github.com/miekg/dns is excellent and is based on the DNS code in the pure Go resolver.

[YanzhaoLi]

Thanks @antong .
Yes. I've done the same test as you did.
For reverse-dns lookup, go resolver did ignore /etc/nsswitch.conf but search /etc/hosts first.
I hit the issue because etcd just uses the go default resolver to do reverse-lookup search for cert check: https://github.com/etcd-io/etcd/pull/8281/files#diff-865eee6d8bcd14cbc443a35074461e1f9295cc30389f7ee43c3f09cfddad4a84R233 .
Currently I can achieve by adjusting my /etc/hosts. I just wonder whether it is right for the way that pure go resolver treat /etc/nsswitch.conf.
For me, it's ok to close this issue.

[cagedmantis]

Closing this at the request of the issue creator. Please feel free to open the issue again if you feel this was closed in error.