Don't use urlsplit on request path

[digitalresistor]

waitress/waitress/parser.py

Line 257 in 94e2311

scheme, netloc, path, query, fragment = urlparse.urlsplit(uri)

This leads to things like:

>>> from urllib import parse as urlparse
>>> urlparse.urlsplit('//testing/whatever')
SplitResult(scheme='', netloc='testing', path='/whatever', query='', fragment='')

Which means we accidentally drop testing before sending it on to the WSGI application. Ask me later how I figured that out.

A request such as:

GET //testing/whatever HTTP/1.1

Is perfectly valid. Non-sensical maybe, but perfectly valid.

[mmerickel]

That just looks like a scheme-relative url. Not sure why you’re passing the path to urlsplit instead of the whole url.

Does pep3333 say servers should collapse / in urls before sending them to the app? Is that why this issue is in waitress?

[digitalresistor]

@mmerickel there is no whole URL when parsing a GET request. You would have to re-constitute a whole URL.

However, the following is ALSO a legal GET request, and is likely why the urlsplit exists in the first place:

GET http://example.com/testing/whatever HTTP/1.0

This form is only ever used when the server is a proxy server, and it is not likely to exist in the wild. But that means both cases do need to get handled.

As for it being a scheme relative URL, it is not, the URL that was accessed by the client was:

http://127.0.0.1//testing/whatever

Which gets turned into this by curl and friends:

GET //testing/whatever HTTP/1.1
Host: 127.0.0.1
User-Agent: curl/7.54.0
Accept: */*

[digitalresistor]

waitress/waitress/parser.py

Lines 200 to 208 in 94e2311

	command, uri, version = crack_first_line(first_line)
	version = tostr(version)
	command = tostr(command)
	self.command = command
	self.version = version
	(self.proxy_scheme,
	self.proxy_netloc,
	self.path,
	self.query, self.fragment) = split_uri(uri)

Specifically this is the uri get from crack_first_line(), we don't even have any information on any other headers yet.

[mmerickel]

ok well if you have some context that says it's in the status line of a request then you know you don't support relative urls so you can normalize it right?

[mmerickel]

I wasn't clear from your original issue that this is specifically about cracking the first line of a request.

[mmerickel]

You have since edited it to be more clear, but I read it before the edits. :p

[digitalresistor]

It was 02:15 in the morning when I first found the issue and added this issue to the issue tracker. I realize that I had a bunch of additional context in my head that was not in the issue. Sorry for not making it more clear.

The issue here is that we crack the first line, and then pass that uri from that to urlsplit. If we are a proxy server then that makes sense, because we will get a full URL and urlsplit will do its magic correctly. If however we have just a path + query string, then urlsplit fails miserably when the path starts with two /'s because it assumes the first part is the netloc.

Basically this needs to change to:

1. See if uri starts with "/", then split on ? and #
2. If not the above, then use urlsplit

[mmerickel]

Is it not enough to check specifically for the uri.startswith('//') situation and drop the first / before passing it to urlsplit?

[digitalresistor]

@mmerickel and then add it back before setting it as the path? It is valid to send //testing/whatever as the PATH_INFO to the WSGI application...