Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: should expand IP address 1.1 to 1.0.0.1 #36822

Open
jamieson99 opened this issue Jan 28, 2020 · 16 comments
Open

net: should expand IP address 1.1 to 1.0.0.1 #36822

jamieson99 opened this issue Jan 28, 2020 · 16 comments

Comments

@jamieson99
Copy link

@jamieson99 jamieson99 commented Jan 28, 2020

nameserver 1.1 in /etc/resolv.conf not parsed

What did you do?

go get github.com/kevinburke/twilio-go

What did you see instead?

go get github.com/kevinburke/twilio-go: module github.com/kevinburke/twilio-go:
Get https://proxy.golang.org/github.com/kevinburke/twilio-go/@v/list:
dial tcp: lookup proxy.golang.org on [::1]:53: dial tcp [::1]:53:
connect: connection refused

Workaround

Change 1.1 to 1.0.0.1 or 1.1.1.1 in /etc/resolv.conf

# /etc/resolv.conf
nameserver 1.1
@davecheney
Copy link
Contributor

@davecheney davecheney commented Jan 28, 2020

Looking at http://man7.org/linux/man-pages/man5/resolv.conf.5.html, name server is defined to be an IPV4 or IPV6 address, nameserver 1.1 appears to be neither so I don't see how it could be used as a DNS server.

Why did you expect that Go would be able to use 1.1 as a nameserver?

@jamieson99
Copy link
Author

@jamieson99 jamieson99 commented Jan 28, 2020

1.1 should expand to 1.0.0.1

Examples:

Try putting nameserver 1.1 into resolv.conf and then ping something; your resolver should properly use that resolver.

Or, just ping 1.1 or traceroute 1.1, or ping 127.1. On Linux (but apparently not Mac), you can even ping 0 (which expands to 0.0.0.0)

CloudFlare's public nameservers are documented at: https://1.1/ (which should be a working link in your browser)

dig google.com @1.1
....
;; ANSWER SECTION:
google.com.     120 IN  A   172.217.6.142
....
;; SERVER: 1.0.0.1#53(1.0.0.1)

The expansion rules are covered here (part of POSIX / IEEE 1003.1).

http://man7.org/linux/man-pages/man3/inet_addr.3.html

#! /usr/bin/env python

import socket
print(socket.gethostbyname('1.1'))
# '1.0.0.1'

It appears that net does not properly expand and parse IPv4 in this format, so that might be the underlying cause here.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 28, 2020

This is part of POSIX (IEEE 1003.1).

Do you have a citation for that? I didn't think that IEEE 1003.1 talked about networking at all. Thanks.

@ianlancetaylor ianlancetaylor changed the title /etc/resolv.conf not interpreted properly net: should treat IP address 1.1 as 1.0.0.1 Jan 28, 2020
@ianlancetaylor ianlancetaylor added this to the Go1.15 milestone Jan 28, 2020
@jamieson99
Copy link
Author

@jamieson99 jamieson99 commented Jan 28, 2020

package main

import (
    "fmt"
    "net"
    "os"
)

func main() {
    ip := net.ParseIP("1.1")
    if ip.To4() == nil {
        fmt.Printf("IPv4 expansion is not working.")
        os.Exit(1)
    }
}

@rhedile
Copy link

@rhedile rhedile commented Jan 28, 2020

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 28, 2020

Thanks for the pointer.

@davecheney
Copy link
Contributor

@davecheney davecheney commented Jan 28, 2020

Indeed, TIL!

@opennota
Copy link

@opennota opennota commented Jan 29, 2020

http://1089054054/

@davecheney
Copy link
Contributor

@davecheney davecheney commented Jan 29, 2020

Well, that's just about all the internet I can handle today. Good evening

@jamieson99 jamieson99 changed the title net: should treat IP address 1.1 as 1.0.0.1 net: should expand IP address 1.1 to 1.0.0.1 Jan 30, 2020
@odeke-em odeke-em removed this from the Go1.15 milestone May 25, 2020
@odeke-em odeke-em added this to the Go1.16 milestone May 25, 2020
@epelc
Copy link

@epelc epelc commented Aug 13, 2020

Are there any security issues related to this potentially changing behavior in higher level packages(net/http, etc)? I think it may be a good idea to make a note about the potential for SSRF and other security problems on the release notes if/when this is changed.

ie if you are filtering some internal address like 192.168.0.2 you could potentially expand via 192.168.2 and a simple filter might not catch it. I know best practice is to use proxy hosts setup on separate networks/subnets but it's been shown in the past with major programs(github enterprise) that many webhook implementations skip this or use alternative methods.

https://www.blackhat.com/docs/us-17/thursday/us-17-Tsai-A-New-Era-Of-SSRF-Exploiting-URL-Parser-In-Trending-Programming-Languages.pdf

@FatalReadError
Copy link

@FatalReadError FatalReadError commented Aug 13, 2020

Good point (and that's an awesome presentation.)

I'm not clear on what exact attack vector you're suggesting here -- are we talking about URL routing with embedded IP's, general input sanitation, other protocols, etc. It seems like most naive implementations that would be expecting an IPv4 at all should also enforce a dotted-quad input sanitation filter, since any developer that would just allow untrusted input without even the bare minimum of bounds checking for some sort of properly formed IP (whatever that means to the developer!) will probably have far larger issues anyway.

Even if the developer has some inscrutable take on what an IPv4 "looks" like, in practice this expansion isn't really much different from how you can drop octets in IPv6, so perhaps the behavior should match whatever is currently done for IPv6 octets.

@iwdgo
Copy link
Contributor

@iwdgo iwdgo commented Nov 8, 2020

One POSIX online reference is on opengroup
Searching for inet_addr returns the page pasted above.

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 8, 2020

Change https://golang.org/cl/268259 mentions this issue: net: expand IP when octets are missing

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Dec 25, 2020

Punting to Go1.17, as per guidance on the CL after the need for clarification.

@odeke-em odeke-em removed this from the Go1.16 milestone Dec 25, 2020
@odeke-em odeke-em added this to the Go1.17 milestone Dec 25, 2020
@AkihiroSuda
Copy link

@AkihiroSuda AkihiroSuda commented Mar 13, 2021

In addition, net.ParseIP() should support hex and octal form.

e.g., "0x7F.010" should correspond to "127.0.0.8".

$ wget http://0x7F.010
--2021-03-13 16:38:50--  http://0x7f.010/
Resolving 0x7f.010 (0x7f.010)... 127.0.0.8
...

@AkihiroSuda
Copy link

@AkihiroSuda AkihiroSuda commented Mar 13, 2021

Punting to Go1.17, as per guidance on the CL after the need for clarification.

The "7.4. Rare IP Address Formats" section of RFC3986 has explanation about the rare formats, though not formal definition: https://tools.ietf.org/html/rfc3986#section-7.4

7.4. Rare IP Address Formats

Although the URI syntax for IPv4address only allows the common
dotted-decimal form of IPv4 address literal, many implementations
that process URIs make use of platform-dependent system routines,
such as gethostbyname() and inet_aton(), to translate the string
literal to an actual IP address. Unfortunately, such system routines
often allow and process a much larger set of formats than those
described in Section 3.2.2.

For example, many implementations allow dotted forms of three
numbers, wherein the last part is interpreted as a 16-bit quantity
and placed in the right-most two bytes of the network address (e.g.,
a Class B network). Likewise, a dotted form of two numbers means
that the last part is interpreted as a 24-bit quantity and placed in
the right-most three bytes of the network address (Class A), and a
single number (without dots) is interpreted as a 32-bit quantity and
stored directly in the network address. Adding further to the
confusion, some implementations allow each dotted part to be
interpreted as decimal, octal, or hexadecimal, as specified in the C
language (i.e., a leading 0x or 0X implies hexadecimal; a leading 0
implies octal; otherwise, the number is interpreted as decimal).

These additional IP address formats are not allowed in the URI syntax
due to differences between platform implementations. However, they
can become a security concern if an application attempts to filter
access to resources based on the IP address in string literal format.
If this filtering is performed, literals should be converted to
numeric form and filtered based on the numeric value, and not on a
prefix or suffix of the string form.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet