Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: reject leading zeros in IP address parsers #30999

Open
opennota opened this issue Mar 22, 2019 · 18 comments
Open

net: reject leading zeros in IP address parsers #30999

opennota opened this issue Mar 22, 2019 · 18 comments

Comments

@opennota
Copy link

@opennota opennota commented Mar 22, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xxx/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/xxx/gocode"
GOPROXY=""
GORACE=""
GOROOT="/home/xxx/go"
GOTMPDIR=""
GOTOOLDIR="/home/xxx/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build040813268=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import "net/http"

func main() {
	http.Get("http://7.7.7.017")
}

What did you expect to see?

7.7.7.017 is interpreted as 7.7.7.15.

$ ping 7.7.7.017
PING 7.7.7.017 (7.7.7.15) 56(84) bytes of data.

What did you see instead?

The program tries to connect to 7.7.7.17.

@agnivade agnivade changed the title net/http: octal literals in IP addresses are interpreted as decimal ones net/url: Parse interprets octal literals in IP addresses as decimal ones Mar 22, 2019
@agnivade
Copy link
Contributor

@agnivade agnivade commented Mar 22, 2019

This is more of a net/url issue rather than net/http.

What does the RFC say regarding this ?

@opennota
Copy link
Author

@opennota opennota commented Mar 22, 2019

I believe that Parse doesn't try to interpret the hostname at all.

@julieqiu julieqiu added this to the Go1.13 milestone Apr 22, 2019
@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@secenv
Copy link

@secenv secenv commented Mar 31, 2021

As shown in this recent article, this behavior could be used in server-side request forgery, local file inclusion and remote file inclusion vulnerabilities.

@tv42
Copy link

@tv42 tv42 commented Mar 31, 2021

There is no real RFC on textual IP address representation. The best we have is https://tools.ietf.org/html/draft-main-ipaddr-text-rep-02 which says

All the forms except for decimal octets are seen as non-standard (despite being quite widely interoperable) and undesirable.

I'd argue Go net.ParseIP/ParseCIDR etc should return an error on zero-prefixed input. It avoids ambiguity and since Go has historically parsed them differently than BSD, an error is a safer change in behavior than silently giving different results.

See also https://man7.org/linux/man-pages/man3/inet_pton.3.html which does not accept zero-prefixed IPs.

@tv42
Copy link

@tv42 tv42 commented Mar 31, 2021

And as I discussed with @secenv on IRC, that article is naive. Typical "attacks" that 0127.0.0.1 enables are enabled also by evil.example.com A 127.0.0.1 in DNS, and the fix for both is to check the target IP after resolving, basically &http.Client{Transport: &http.Transport{DialContext: dialOnlySafeIPs}}

@secenv
Copy link

@secenv secenv commented Mar 31, 2021

I forgot to add that it is indeed an issue that affects net.ParseCIDR https://play.golang.org/p/HpWqhr9tZ53 . I agree with @tv42, those functions should return errors. The documentation should at least warn the developer.

Guess I should mention @FiloSottile for further discussion on the security impact of this issue.

@tv42
Copy link

@tv42 tv42 commented Mar 31, 2021

I found a more authoritative RFC on IP address textual representation -- although it's only Informational not Standards Track: https://tools.ietf.org/html/rfc6943#section-3.1.1

Since Go doesn't use the "loose" syntax of RFC6943, it's non-conforming already. Rejecting non-dotted-decimal inputs would make Go use the "strict" syntax.

@rsc
Copy link
Contributor

@rsc rsc commented Apr 5, 2021

I agree about changing Go's IP address parsers
(ParseIP, ParseCIDR, any others) to reject leading zeros (except "0"),
because:

(1) the RFCs are mostly quiet but in a few places hint that decimal is the right interpretation,
(2) Go interprets leading zeros as decimal, and
(3) BSD stacks nonetheless interpret leading zeros as octal.
(4) The fact that basically no one has noticed this divergence implies
that essentially no one uses leading zeros in IP addresses.

It seems like an open question whether this should be done
in a point release or saved for the next major release (Go 1.17).
But to start, we should agree to do it at all.
Adding to the proposal process.

@rsc rsc changed the title net/url: Parse interprets octal literals in IP addresses as decimal ones net/url: reject leading zeros in IP address parsers Apr 5, 2021
@rsc rsc changed the title net/url: reject leading zeros in IP address parsers proposal: net/url: reject leading zeros in IP address parsers Apr 5, 2021
@rsc rsc modified the milestones: Backlog, Proposal Apr 5, 2021
@rsc rsc added this to Incoming in Proposals Apr 5, 2021
@rsc rsc added the Proposal label Apr 5, 2021
@FiloSottile
Copy link
Contributor

@FiloSottile FiloSottile commented Apr 6, 2021

And as I discussed with @secenv on IRC, that article is naive. Typical "attacks" that 0127.0.0.1 enables are enabled also by evil.example.com A 127.0.0.1 in DNS, and the fix for both is to check the target IP after resolving, basically &http.Client{Transport: &http.Transport{DialContext: dialOnlySafeIPs}}

I find this pretty convincing, especially given that net.Dial and net.Listen will parse the IPs as decimal.

To end up vulnerable due to this mismatch, an application would have to parse the IP with Go, reject any hostnames, apply security-relevant logic to the return value, and then pass the input (not the encoding of the return value) to a different, non-Go application which is happy to parse the IP as octal.

Generally, this is another instance where relying on parser alignment instead of re-encoding outputs is a fragile design.

We are not aware of any application for which this leads to a security issue, if anyone does please let us know at security@golang.org as that would help evaluate whether to backport the fix.

In any case, I definitely agree we should just consider these inputs invalid in Go 1.17.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Apr 7, 2021

Related: #43389 ("net: limit the size of ParseIP input?")

@rsc
Copy link
Contributor

@rsc rsc commented Apr 7, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Apr 7, 2021
@FiloSottile FiloSottile changed the title proposal: net/url: reject leading zeros in IP address parsers proposal: net: reject leading zeros in IP address parsers Apr 8, 2021
@liggitt
Copy link
Contributor

@liggitt liggitt commented Apr 8, 2021

One notable use of the stdlib methods is to validate API fields expected to contain IP / CIDR data. When stdlib methods change to reject data they previously accepted, that makes previously validated and persisted data turn invalid when checked by the same validation code.

While I agree with the goals of bounding input and ensuring data handled by diverse implementations is unambiguous, there should be a reasonable migration path for callers using stdlib methods for this purpose. What would we expect these callers to do before/after the proposed stdlib changes to move away from accepting 0-prefixed IP octets in new data, and to start detecting/migrating existing data?

Options that come to mind:

  • Prevalidate IP/CIDR inputs to detect/reject leading 0's in addition to calling stdlib Parse methods? This seems like it reimplements bits of IP parsing, which is fragile/complex, especially for ipv6 addresses (which was why stdlib methods were used in the first place)
  • Require normalized form (e.g. ParseIP().String())? This seems burdensome for ipv6 addresses, which are quite flexible and make many accommodations to simplify textual expression
@rsc
Copy link
Contributor

@rsc rsc commented Apr 14, 2021

Prevalidate IP/CIDR inputs to detect/reject leading 0's in addition to calling stdlib Parse methods? This seems like it reimplements bits of IP parsing, which is fragile/complex, especially for ipv6 addresses (which was why stdlib methods were used in the first place)

This seems fine to me if really needed. Instead of reimplementing those bits, though, just make a copy of the existing standard library routines. The license permits that.

The problem with "validate API fields" here is that if the validation disagrees with the eventual use, the validation is wrong. It's probably better for everyone involved to reject those than to mutually misunderstand them.

@thockin
Copy link

@thockin thockin commented Apr 15, 2021

That's rather unpleasant. We have persisted data that we don't control and can't ever really declare as invalid. We'll have to allow old uses of the "sloppy" format in perpetuity.

No hope of adding a ParseIPSloppy() sibling in the stdlib ?

@rsc
Copy link
Contributor

@rsc rsc commented Apr 21, 2021

If Kubernetes needs ParseIPDecimal then I would suggest adding that to Kubernetes rather than the standard library.
Note that such persisted data does not have the meaning Kubernetes thinks it does when it ends up in non-Go programs.
For example Curl: https://daniel.haxx.se/blog/2021/04/19/curl-those-funny-ipv4-addresses/.

@thockin
Copy link

@thockin thockin commented Apr 21, 2021

@rsc
Copy link
Contributor

@rsc rsc commented Apr 21, 2021

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Accept in Proposals Apr 21, 2021
@rsc rsc moved this from Likely Accept to Accepted in Proposals Apr 28, 2021
@rsc
Copy link
Contributor

@rsc rsc commented Apr 28, 2021

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: net: reject leading zeros in IP address parsers net: reject leading zeros in IP address parsers Apr 28, 2021
@rsc rsc modified the milestones: Proposal, Backlog Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Accepted
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
12 participants