dial tcp: lookup ..... no such host ==> DNS doesn't work

[0x8f701]

What version of Go are you using (go version)?

$ go version
go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cheng/.cache/go-build"
GOENV="/home/cheng/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/cheng/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build426061178=/tmp/go-build -gno-record-gcc-switches"

What did you do?

http get and parse html, extract their href

// Package util provides ...
package util

import (
	"errors"
	"fmt"
	"net/http"
	"strings"

	"github.com/PuerkitoBio/goquery"
)

func DiscoverVersions(baseUrl string) ([]string, error) {
	if !strings.HasSuffix(baseUrl, "/") {
		baseUrl = fmt.Sprintf("%s/", baseUrl)
	}

	res, err := http.Get(baseUrl)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != 200 {
		errMsg := fmt.Sprintf("status code error: %s", res.Status)
		return nil, errors.New(errMsg)
	}

	doc, err := goquery.NewDocumentFromReader(res.Body)
	if err != nil {
		return nil, err
	}

	var versions []string
	doc.Find("ul li a").Each(func(i int, s *goquery.Selection) {
		if href, exists := s.Attr("href"); exists {
			versions = append(versions, href)
		}
	})
	return versions, nil
}

It gives me error:

2020-09-16T09:47:51.469+0200    ERROR   version         https://username:password@example.com/software/release/, error: Get " https://username:password@example.com/software/release": dial tcp: lookup example.com: no such host

What did you expect to see?

DNS lookup for http get should work for most of the time

What did you see instead?

DNS lookup fails frequently

[mdlayher]

Please see https://github.com/golang/go/wiki/Questions. Closing.

[0x8f701]

@mdlayher isn't it a bug? I haven't seen this in any other languages using a simple Http request...

[ianlancetaylor]

Hundreds of thousands of people are able to use Go to make TCP connections successfully. If you really think this is a bug in Go, and not in your local DNS server, then show us a complete, self-contained Go program that demonstrates the problem. The error message you show above is not coming from the code you show above, so there is nothing here for us to investigate.

But I recommend you sart with a forum, as @mdlayher suggested.

[0x8f701]

hi @ianlancetaylor I already showed all the code that I used. I know it's hard to reproduce because even on my side it just happens 1 out of 100 probably, but at lease please just search in the code using the error message that I provided, if go don't want to do real opensourcing then please just make it private.

I'm not a person who would like to waste time creating junk issue for go, I used go for openwrt devices, it worked quite well. I made this issue because this error happened during my important demo with a super good internet. It then happened again on production server.

That's why I think there must be a bug

[0x8f701]

[0x8f701]

@ianlancetaylor @mdlayher I made a simple gif to demonstrate go's http get's problem, I've got really too much timeout error.

You can see nodejs's axios performs much better than go http get, it doesn't have any timeout error at the beginning, at last it does get timeout errors but I think that's maybe caused by limited resources or other reason.

[0x8f701]

test.go

// Package main provides ...
package main

import (
	"errors"
	"fmt"
	"net/http"
	"strings"
	"time"
)

func DiscoverVersions(baseUrl string) ([]string, error) {
	if !strings.HasSuffix(baseUrl, "/") {
		baseUrl = fmt.Sprintf("%s/", baseUrl)
	}

	res, err := http.Get(baseUrl)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != 200 {
		errMsg := fmt.Sprintf("status code error: %s", res.Status)
		return nil, errors.New(errMsg)
	}

	return []string{}, nil
}

func main() {
	for {
		_, err := DiscoverVersions("https://fr-cloud.ubudu.com")
		if err != nil {
			fmt.Printf("error %v\n", err)
		}
		time.Sleep(1 * time.Second)
	}
}

test.js

const axios = require("axios");

setInterval(() => {
    axios.get("https://fr-cloud.ubudu.com/").catch(console.log);
}, 1000);

[0x8f701]

However, I cannot reproduce the DNS lookup no such host error, I'll try to make it reproducible once got time. I'm sure there is a bug because with other languages I never ran into that much DNS lookup issues

[ianlancetaylor]

See the discussion of "Name Resolution" at https://golang.org/pkg/net. See if setting the environment variable GODEBUG=netdns=cgo helps.

[t9t]

Hello, I have some information to add to this. Please check if it should be reopened if it appears to actually be a bug and/or if I should open a new issue.

I narrowed it down to this issue occurring when accessing a "private" domain (I don't know the correct terminology) on our corporate network reachable over VPN, when using a binary on macOS which was compiled on Linux (or rather, in the golang:1.15 Docker container in my specific case).

I observed the following (all when running the binary on macOS):

Situation	Result
Public URL, eg. api.github.com	Just works
Private domain, compiled on Mac	Works fine
Private domain, compiled on Mac, GODEBUG=netdns=cgo	Works fine
Private domain, compiled on Mac, GODEBUG=netdns=go	no such host
Private domain, compiled on Linux	no such host, no matter the GODEBUG=netdns=.. setting

The full error message in my case is:

Get "https://private-domain...": dial tcp: lookup private-domain... on 192.168.1.1:53: no such host

This is quite a problem for us, because we have some internal tools that we run on our Macbooks, but compile in Docker containers.

Please let me know if there are any additional steps I can take to provide even more information, especially to determine if this is actually a bug in Go, or a corrupt DNS setup.

[seankhliao]

have you verified that it actually uses your selected resolver of go/cgo? GODEBUG=netdns=cgo+1

[t9t]

have you verified that it actually uses your selected resolver of go/cgo? GODEBUG=netdns=cgo+1

I hadn't, but doing this provided some more insight indeed. With the Mac built binary, it says using cgo DNS resolver and GODEBUG setting forcing use of Go's resolver (same results as above, works with cgo, doesn't work with go). With the Linux (Docker) built binary, in either case it just says built with netgo build tag; using Go's DNS resolver. Note that I didn't specify that explicitly, I literally only did GOOS=darwin GOARCH=amd64 go build main.go.

Though I guess it makes sense that when it's built on Linux, it can't use cgo when running on Mac? When I add -tags netcgo to the build, it says /usr/local/go/src/net/cgo_bsd.go:15:72: could not determine kind of name for C.AI_MASK. But perhaps now it's yet becoming more of a discussion rather than a bug report.

Let me know if I can provide any additional information for this issue.