Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: Error when running concurrent DNS queries using cgo on MacOS #30521

Closed
cezarsa opened this issue Mar 1, 2019 · 8 comments

Comments

Projects
None yet
5 participants
@cezarsa
Copy link
Contributor

commented Mar 1, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes, only on the latest release, it wasn't a problem with Go 1.11

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/cezarsa/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/cezarsa/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/18/22s9w_gd21979c67k0p0p7t00000gn/T/go-build808727368=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

What did you expect to see?

With the code available in https://play.golang.org/p/Za2kDnfq-wB on MacOS Mojave 10.14

The expected result for running it with both no environment variables and with PARALLEL=1 would be:

$ go run dns.go
remote addr for udp4: 127.0.0.1:27017
remote addr for udp6: [::1]:27017

What did you see instead?

Running it with no envs yields the expected result. However, running it with PARALLEL=1 causes the following output:

$ PARALLEL=1 go run dns.go
ERR for udp4: dial udp4: address localhost: no suitable address found
remote addr for udp6: [::1]:27017

The problem only seems to exist when running on MacOS, and only when using cgo resolver, using Go resolver or on Linux does not causes any errors:

$ GODEBUG=netdns=cgo+1 PARALLEL=1 go run dns.go
go package net: using cgo DNS resolver
ERR for udp4: dial udp4: address localhost: no suitable address found
remote addr for udp6: [::1]:27017
$ GODEBUG=netdns=go+1 PARALLEL=1 go run dns.go
go package net: GODEBUG setting forcing use of Go's resolver
remote addr for udp4: 127.0.0.1:27017
remote addr for udp6: [::1]:27017
@cezarsa

This comment has been minimized.

Copy link
Contributor Author

commented Mar 1, 2019

I've been taking a look at the lookup code, and it looks like the problem is here:

ch, called := r.getLookupGroup().DoChan(host, func() (interface{}, error) {

The lookup group will only allow 1 query to run even when they are using different values for the network protocol, and the underlying lookup implementation is only returning ip addresses valid to the chosen network.

A possible fix would be to change the key for the lookup group to be network + host, instead of only host. Testing locally this change does fix the problem, but I don't know if it's a valid solution or some other implications unknown to me of such change.

Also, from the Git history it looks like this didn't change for 1.12 so this does not explain why this used to work on 1.11.

@bcmills bcmills changed the title Error when running concurrent DNS queries using cgo on MacOS net: Error when running concurrent DNS queries using cgo on MacOS Mar 1, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 1, 2019

@bcmills bcmills added this to the Go1.13 milestone Mar 1, 2019

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Mar 1, 2019

Changing the key to network + "\000" + host seems fine to me. It would be nice to have a test, though.

@cezarsa

This comment has been minimized.

Copy link
Contributor Author

commented Mar 2, 2019

@ianlancetaylor Cool, if no one is working on this I'd like to try sending a CL including some tests. Further tracking down the problem I think the source of the regression is actually c659be4 but I think changing the lookup group key still makes more sense as a fix.

@gopherbot

This comment has been minimized.

Copy link

commented Mar 7, 2019

Change https://golang.org/cl/166037 mentions this issue: net: use network and host as singleflight key during lookupIP

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2019

@gopherbot please consider this for backport to 1.12

@gopherbot

This comment has been minimized.

Copy link

commented Mar 26, 2019

Backport issue(s) opened: #31062 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

As reported in #31058, this is a regression in Go 1.12 and there's no workaround for Windows and others that have no support for GODEBUG=netdns=go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.