Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
net: /etc/hosts parsing truncates to 4KB (os.GetPagesize) #21674
Due to the way
Go silently truncates the data after 4KB and does not parse lines on the threshold, since EOL is never reached for them.
The cgo equivalents for this functionality handle these files properly, and so should Go.
We often see these large hosts files in the web hosting environment (e.g. cPanel), when servers are behind NAT and require large hosts files.
Env: Go1.9, linux/amd64
I have also attached a Dockerfile tarball that reproduces the issue fully - go-4kb-hosts.tar.gz
@rsc I suppose that the seemingly arbitrary os.Getpagesize() limit was set because we wanted a performance hit from reads on page boundaries or that we thought that size was sufficient? I don't see a file limit for a hosts file in either of the below:
I've been tracking the CL and I'm not sure how to add my input, sorry if I'm doing it wrong.
I saw a comment by @rsc that worries me a little:
I originally ran into this issue with customers exactly because of entries on specific lines that exceeded 4kB (big NAT-ted cPanel servers). If the system resolver deals with it without a problem I would not expect Go to introduce such a surprising and arbitrary limitation here.
I apologise if the original issue made it look like the only concern was the remainder of the file - properly parsing long lines is equally important to my application.