Repository navigation
net: isDomainName rejects valid domains #17659
Description
isDomainName is limited to hostname-compatible "preferred name" LDH labels, with an exception made for underscores per #1167. But RFC 2181 is clear about any octet being valid in a DNS label:
Those [length] restrictions aside, any binary string whatever can be used as the label of any resource record.
What version of Go are you using (go version)?
go1.7.1
What operating system and processor architecture are you using (go env)?
linux/amd64
GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build693750050=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
What did you do?
query uncommon domain names
package main
import (
"fmt"
"net"
)
func main() {
domains := []string{
`*.golang.org`,
`\.\.\..golang.org`,
`\065.golang.org`,
`\000.golang.org`,
}
for _, domain := range domains {
fmt.Printf("\n%s\n", domain)
fmt.Println(net.LookupHost(domain))
}
}What did you expect to see?
successful DNS queries
*.golang.org
[216.58.219.209 2607:f8b0:4006:80e::2011] <nil>
\.\.\..golang.org
[216.58.219.209 2607:f8b0:4006:80e::2011] <nil>
\065.golang.org
[216.58.219.209 2607:f8b0:4006:80e::2011] <nil>
\000.golang.org
[216.58.219.209 2607:f8b0:4006:80e::2011] <nil>
What did you see instead?
valid names were rejected
*.golang.org
[] lookup *.golang.org: invalid domain name
\.\.\..golang.org
[] lookup \.\.\..golang.org: no such host
\065.golang.org
[216.58.219.209 2607:f8b0:4006:80e::2011] <nil>
\000.golang.org
[] lookup \000.golang.org: no such host
Activity
quentinmit commentedon Nov 1, 2016
/cc @rsc in case he disagrees
rsc commentedon Nov 1, 2016
The hard part here is that it seems to be the case that not any binary name should be accepted. Otherwise thing like Punycode would not exist. What is the dividing line for what can be looked up directly and what needs special processing?
rsc commentedon Nov 1, 2016
/cc @mdempsky @mikioh @mpvl @pmarks-net
cznic commentedon Nov 1, 2016
Not according to RFC1035. However, RFC 2181 updates RFC1035 in the way @gibson042 reported, ie. a label can consist of any octets. RFC4343, Section 2 discusses handling and escaping non-ASCII labels.
mikioh commentedon Nov 16, 2016
https://tools.ietf.org/html/rfc6055#section-3 shows some discussion on the use of non-LDH labels in DNS.
bradfitz commentedon Jun 13, 2017
Thanks for the links, but looks like there's still no decision here. Bumping to Go 1.10.
/cc @mdempsky
gopherbot commentedon Oct 27, 2017
Change https://golang.org/cl/73830 mentions this issue:
net: recognize non-LDH domain names as valid per RFC 1035gibson042 commentedon Oct 27, 2017
I finally got around to fixing this—the code should clarify any open questions, but I'm also happy to address them directly.
https://go-review.googlesource.com/c/go/+/73830
rsc commentedon Dec 4, 2017
It looks like we are still waiting on a proposed answer for the question above: "What is the dividing line for what can be looked up directly and what needs special processing?"
Can anyone give an authoritative answer or at least make a good suggestion?
/cc @mdempsky @mikioh @mpvl @pmarks-net
12 remaining items
bradfitz commentedon Sep 24, 2018
@snowcrumble, I sent https://go-review.googlesource.com/c/go/+/136900 for that.
gopherbot commentedon Sep 24, 2018
Change https://golang.org/cl/136900 mentions this issue:
net: don't reject domain names with only numbers and hyphensnet: don't reject domain names with only numbers and hyphens
mdempsky commentedon Oct 31, 2018
RFC 1035 § 5 is about master files, which package net does not support.
Is there another RFC that extends the requirement for supporting \DDD escapes to client resolver libraries like package net?
gibson042 commentedon Nov 1, 2018
Over the years, the ASCII syntax defined in that section has been generalized into what is now known as "presentation format"¹. As far as I know, it is the only valid text-safe means of representing an arbitrary domain name. When you say "package net does not support" it, I think you mean that it supports only a subset—and if that is indeed intentional, then the gap should be documented and the error messages should be normalized and corrected.
But that seems like a mistake to me... what are the reasons for not allowing
Lookupmethods to resolve arbitrary domain names?¹ In the context of DNS, the earliest use of that term I could find is in in RFC 3658 (before that it was still "Master File Format" as in RFC 1706, with a transition beginning sometime around RFC 2535).
bradfitz commentedon Nov 1, 2018
I haven't been following this bug closely, but:
Obviously we weren't trying to make Lookup fail to resolve arbitrary domain names. More than likely we tried to add some validation somewhere and got it wrong and ended up too aggressive.
bradfitz commentedon Nov 14, 2018
I saw this guide to modern DNS recently which ignores all the old parts of DNS RFCs which are no longer accurate:
https://powerdns.org/hello-dns/
Maybe the answer to this question is in there somewhere.
gibson042 commentedon Nov 16, 2018
Punycode exists because although any octet is permissible in a DNS label, there is no guidance on how to interpret octets 128 through 255. It is definitively not acceptable to assume that they are UTF-8, and any domain name including them, although valid, is not an Internet host name because it does not conform to the letter-digit-hyphen "preferred name syntax" (but note that underscore names such as
_jabber._tcp.example.comdo not conform to it either—they are not "host names" but nonetheless need to remain resolvable, which is why Go already makes an exception as of 2408a4b to explicitly permit underscores). Punycode is a non-UTF-8 encoding of non-ASCII strings into ASCII, with the express purpose of representing internationalized domain names as true LDH host names.But all of that, while interesting for background purposes, does not change the need to respect arbitrary octets in domain name labels as required by RFC 2181. I really don't know how to be more clear in demonstrating that such is the case.