Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: in LookupHost, DNS resolves addresses with dot using search domain without it being specified in resolv.conf #31934

Open
XANi opened this issue May 9, 2019 · 7 comments

Comments

@XANi
Copy link

commented May 9, 2019

Version: go version go1.11.6 linux/amd64, tested also on 1.10

Issue: *.local addresses are not sent to resolver set by /etc/resolv.conf

Background: Kuberneres clusters use cluster.local as default domain for their internal DNS service. So test code:

package main

import (
	"fmt"
	"net"
)

func main() {
	addr, err := net.LookupHost("bot.default.svc.cluster.local")
	fmt.Printf("%+v %s", addr, err)
}
$ go run 1.go
[127.0.0.1] %!s(<nil>)%  

resolv.conf:

nameserver 10.0.116.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

request using system resolver:

$ dig +short @10.0.116.10 qubebot.default.svc.cluster.local
10.0.117.84

@andybons andybons added this to the Unplanned milestone May 13, 2019

@andybons

This comment has been minimized.

Copy link
Member

commented May 13, 2019

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 13, 2019

Can you try Go 1.12 too?

But I'm not sure how to repro this myself. In your dig example, did you mean qubebot or bot?

Previously: #16739 but that looks old (Go 1.7-ish).

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 13, 2019

I kubectl exec'ed into a pod in a Kubernetes cluster and can't reproduce. Looks fine to me:

root@mon-655d77c679-wd5v8:/# cat /etc/resolv.conf 
search default.svc.cluster.local svc.cluster.local cluster.local ward.fitzpat.com
nameserver 10.96.0.10
options ndots:5
root@mon-655d77c679-wd5v8:/# dig +short @10.96.0.10 mon.default.svc.cluster.local
10.96.184.136

root@mon-655d77c679-wd5v8:/# ping mon.default.svc.cluster.local
PING mon.default.svc.cluster.local (10.96.184.136) 56(84) bytes of data.
^C

root@mon-655d77c679-wd5v8:/# ./lookup mon.default.svc.cluster.local
[10.96.184.136]

root@mon-655d77c679-wd5v8:/# ./lookup mon
[10.96.184.136]

root@mon-655d77c679-wd5v8:/# GODEBUG=netdns=go+1 ./lookup mon
go package net: GODEBUG setting forcing use of Go's resolver
[10.96.184.136]

@XANi

This comment has been minimized.

Copy link
Author

commented May 15, 2019

But I'm not sure how to repro this myself. In your dig example, did you mean qubebot or bot

qubebot was just name of app I've encountered it in, name doesn't matter as long as .local is here, you can just try test.local

Can you try Go 1.12 too?

Sure, debian package:

ᛯ /usr/lib/go-1.12/bin/go run /tmp/1.go 
[127.0.0.1] %!s(<nil>)%
ᛯ cat /tmp/1.go                        
package main

import (
	"fmt"
	"net"
)

func main() {
	addr, err := net.LookupHost("test.local")
	fmt.Printf("%+v %s", addr, err)
}

and from webpage:

ᛯ /tmp/go/bin/go run /tmp/1.go
[127.0.0.1] %!s(<nil>)
ᛯ /tmp/go/bin/go version                                                                                                                                     
go version go1.12.5 linux/amd64

in all cases tested for test.local domain.

Also, it only happens for LookupHost, LookupAddr is ok:

ᛯ cat /tmp/2.go
package main

import (
	"fmt"
	"net"
)

func main() {
	addr, err1 := net.LookupAddr("test.local")
	host, err2 := net.LookupHost("test.local")
	fmt.Printf("LookupAddr: %+v %s\n", addr, err1)
	fmt.Printf("LookupHost: %+v %s\n", host, err2)
}
ᛯ go run /tmp/2.go
LookupAddr: [] lookup test.local: unrecognized address
LookupHost: [127.0.0.1] %!s(<nil>)

I have updated the title to straighten that up

@XANi XANi changed the title net: DNS resolves .local addresses without asking DNS server net.LookupHost: DNS resolves .local addresses without asking DNS server May 15, 2019

@bradfitz

This comment has been minimized.

Copy link
Member

commented May 15, 2019

Show the output of running with environment GODEBUG=netdns=2.

Also, what is your /etc/hosts and /etc/nsswitch.conf and /etc/resolv.conf?

@XANi

This comment has been minimized.

Copy link
Author

commented May 17, 2019

-> ᛯ GODEBUG=netdns=2 /usr/lib/go-1.12/bin/go run /tmp/1.go
go package net: dynamic selection of DNS resolver
go package net: hostLookupOrder(test.local) = cgo
[127.0.0.1] %!s(<nil>)       

tested with empty nsswitch (empty kubernetes container with static Go binary) and with

passwd:         compat
group:          compat
shadow:         compat

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

did you ran my examples ? what do they return on your machine ?

@XANi

This comment has been minimized.

Copy link
Author

commented May 17, 2019

Ok, I think I found the problem:
Looking at pcaps of DNS traffic :

System resolver asks for test.local. It stops there because there is a dot in the name.

Go asks for test.local then asks for test.local + search domain (say test.local.example.com)

the difference is that system resolver only adds search domain if domain does not contain a dot. the ndots setting of resolv.conf controls that (from man resolv.conf):

Resolver queries having fewer than ndots dots (default is 1) in them will be attempted using each component of the search path in turn until a match is found.

so default only uses search domains if there is no dot in name, while Go disregards that and queries search domains regardless of what is in the name

And the reason it happens on my setup is because one of my predecessors added CNAME *.local.example.com -> local.example.com (for some goddamn reason...) and A local.example.com 127.0.0.1 which means when I tried to resolve any.name.dot.local it resolved to 127.0.0.1...

Here is a pcap with the problem:

go.pcap.gz

domain somedomain.com
search somedomain.com
nameserver 8.8.8.8

@XANi XANi changed the title net.LookupHost: DNS resolves .local addresses without asking DNS server net.LookupHost: DNS resolves addresses with dot using search domain without it being specified in resolv.conf May 26, 2019

@dmitshur dmitshur changed the title net.LookupHost: DNS resolves addresses with dot using search domain without it being specified in resolv.conf net: in LookupHost, DNS resolves addresses with dot using search domain without it being specified in resolv.conf Jul 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.