net: Go DNS resolver does not read /etc/hosts

[ghost]

What version of Go are you using (go version)?

go version go1.9.2 linux/amd64

What did you do?

The following script creates a Docker container with a cgo-disabled Go binary that resolves a host defined in the /etc/hosts file.

#!/bin/bash
set -e
set -x
cat > Dockerfile <<EOF
FROM alpine:3.6

COPY internal.sh /tmp/internal.sh
COPY go_dns /tmp/go_dns

CMD /tmp/internal.sh
EOF

cat > go_dns.go <<EOF
package main

import (
	"net"
	"fmt"
)

func main() {
	fmt.Println(net.LookupHost("example.com"))
}
EOF

cat > internal.sh <<EOF
set -e
set -x

export GODEBUG=netdns=10

cat /etc/resolv.conf

cat /etc/hosts

/tmp/go_dns

nslookup example.com
EOF

chmod +x internal.sh

go version
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o go_dns go_dns.go
docker build --no-cache -t go_dns .
docker run --rm \
           --add-host example.com:192.168.2.50 \
           -i \
           -t \
           go_dns

What did you expect to see?

Go's DNS resolver resolve the address that is contained in /etc/hosts.

Output:

+ cat
+ cat
+ cat
+ chmod +x internal.sh
+ go version
go version go1.9.2 linux/amd64
+ GOOS=linux
+ GOARCH=amd64
+ CGO_ENABLED=0
+ go build -o go_dns go_dns.go
+ docker build --no-cache -t go_dns .
Sending build context to Docker daemon 2.691 MB
Step 1 : FROM alpine:3.6
 ---> 053cde6e8953
Step 2 : COPY internal.sh /tmp/internal.sh
 ---> 58c4e4eabbd7
Removing intermediate container e915818d62ae
Step 3 : COPY go_dns /tmp/go_dns
 ---> bcf50d520610
Removing intermediate container 85918214a98c
Step 4 : CMD /tmp/internal.sh
 ---> Running in cf4216686f20
 ---> c75e5b4ccec8
Removing intermediate container cf4216686f20
Successfully built c75e5b4ccec8
+ docker run --rm --add-host example.com:192.168.2.50 -i -t go_dns
+ export GODEBUG=netdns=10
+ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
#nameserver 127.0.1.1
+ cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
192.168.2.50	example.com
172.17.0.2	b44f6648150b
+ /tmp/go_dns
go package net: built with netgo build tag; using Go's DNS resolver
go package net: hostLookupOrder(example.com) = dns,files
[93.184.216.34 2606:2800:220:1:248:1893:25c8:1946] <nil>
+ nslookup example.com
nslookup: can't resolve '(null)': Name does not resolve

Name:      example.com
Address 1: 192.168.2.50 example.com

What did you see instead?

An external DNS lookup was performed.

(This may be the same issue that was happening here #14170)

[ghost]

Reading the source of net/conf.go, I see the following comment in regards to /etc/nsswitch.conf:

// glibc says the default is "dns [!UNAVAIL=return] files"
// http://www.gnu.org/software/libc/manual/html_node/Notes-on-NSS-Configuration-File.html.

It turns out that file does not exist in the container. If I create it with the following contents:

hosts: files dns

I get the expected result.

[Kyslik]

Thanks a ton!

https://gitlab.com/gitlab-org/gitlab-runner/issues/2959

[deepakchoudhary]

Docker container with a cgo-disabled Go binary that needs to resolves a host defined in the /etc/hosts file but wasn't working
@bontibon 's solution to worked for me
inside container run
echo "hosts: files dns" > /etc/nsswitch.conf

[Barbery]

Hi, how to fix the problems in mac osx? I can not create the /etc/nsswitch.conf file

[kfox1111]

As this is a common deployment method with golang and k8s with minimal images, this has very unexpected results. Can we fix this to function as expected in a minimal environment?

@thockin

[kfox1111]

Ping

[panamafrancis]

@kfox1111 You should just change your base image, golang should always defer to /etc/nsswitch.conf
@Barbery Here is not the place for sysadmin questions (hint: use sudo)

[qinghai5060]

another reason could be the permission ot '/etc/hosts', it should be 644 when user is not root

[kfox1111]

@panamafrancis it is common to use alpine as a base image as its size is small. it uses a non glibc implementation and does not contain nsswitch.conf. The distro assumes nonglibc behavior as its not glibc based, and behaves well in the absence of nsswitch.conf. golang chose to implement glibc's failback behaviour in the absence of nsswitch.conf, which IMO is a bad default. etc/hosts should be honored by default in the absence of nsswitch.conf. Alpine is not going away any time soon, and convincing folks not to use alpine is going to be a hard sell. I'd suggest golang change its default to a safer one, ie honor etc/hosts first, in light of this. This would be the least surprise option as well as one that hardens security.

[kfox1111]

Ok... just hit this again when switching to k8s 1.12.