Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Fix for files and directories with + as a character in the file name b0rking Rack::Directory #265

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
7 participants

Having a + in file and path names is a perfectly cromulent thing, and unescaping the name magically turns those + es into es

VictorLowther added some commits Nov 16, 2011

@VictorLowther VictorLowther Unescaping PATH_INFO transforms /file_name.file on the filesystem to …
…/file name.file from the POV of the web server, which is a Bad Thing.
245b420
@VictorLowther VictorLowther Unescaping PATH_INFO transforms /file+name.file on the filesystem to …
…/file name.file from the POV of the web server, which is a Bad Thing.
290cbe7
Contributor

josh commented Nov 16, 2011

+s in urls mean spaces.

@josh josh closed this Nov 16, 2011

On Tue, Nov 15, 2011 at 11:38 PM, Joshua Peek
reply@reply.github.com
wrote:

+ in urls mean spaces.

I am aware of that. '+' in a filename does not mean a space, however,
and Rack should respect that. Ditching the unescape fixed that issue
for me locally.


Reply to this email directly or view it on GitHub:
#265 (comment)

Contributor

richmeyers commented Nov 16, 2011

+ is a reserved character in URLs, whoever is generating a URL with unencoded + needs to encode +.

http://tools.ietf.org/html/rfc3986#section-2.2

On Nov 16, 2011 8:53 AM, "Rich Meyers" <
reply@reply.github.com>
wrote:

  • is a reserved character in URLs, whoever is generating a URL with
    unencoded + needs to encode +.

http://tools.ietf.org/html/rfc3986#section-2.2

Then that whoever is Rack::Directory.

If you want a demo, configure a racked webserver to serve an Ubuntu install
CD using Rack::Directory, and note the failure to be able to actually
download any file with a + in the name.


Reply to this email directly or view it on GitHub:
#265 (comment)

Contributor

lgierth commented Nov 16, 2011

Maybe that's your HTTP client not escaping + as it thinks it's a space-plus? Rack should definitely stick to the URI spec.

Contributor

richmeyers commented Nov 16, 2011

@VictorLowther it does look like Rack::Directory is not encoding file names, and that would be the bug. The correct fix would be to have Rack::Directory encode file names in directory listings.

A but more background:

I encountered this bug when trying to use Rainbows to serve up an Ubuntu install tree for net installs. Out of all the web servers I tried, only Rainbows failed to act as a backing webserver when using Rack::Directory to serve up the directory tree -- the install failed the first time it tried to serve a filename with a + in the name.

To replicate:
sudo gem install rack rainbows
mkdir -p test/foo+bar
echo "here"> test/foo+bar/bar+baz
echo 'run Rack::Directory.new(".")' >test/config.ru
(cd test; rainbows -p 8080)

Then, in a different terminal:
wget -O - --save-headers http://localhost:8080/
wget -O - --save-headers http://localhost:8080/foo+bar/
wget -O - --save-headers http://localhost:8080/foo+bar/bar+baz

wget will print a directory index for the first pull, and get 404 errors for the second and third. The Rainbows output will show:

127.0.0.1 - - [19/Nov/2011 09:27:45] "GET / HTTP/1.0" 200 - 0.0153
127.0.0.1 - - [19/Nov/2011 09:28:14] "GET /foo+bar/ HTTP/1.0" 404 28 0.0024
127.0.0.1 - - [19/Nov/2011 09:31:40] "GET /foo+bar/bar+baz HTTP/1.0" 404 35 0.0023

With the changes in this patch, I get:

127.0.0.1 - - [19/Nov/2011 09:39:55] "GET / HTTP/1.0" 200 - 0.0046
127.0.0.1 - - [19/Nov/2011 09:40:00] "GET /foo+bar/ HTTP/1.0" 200 - 0.0046
127.0.0.1 - - [19/Nov/2011 09:40:07] "GET /foo+bar/bar+baz HTTP/1.0" 200 - 0.0023

Webrick, nginx, thttpd, and apache all do the expected thing. The issue is not whether the client is failing to properly encode requests to the server (unless wget has managed to get it wrong for all these years). The issue is that Rack appears to be doing the Wrong Thing when decoding the requests and matching them against the files I am asking it to serve.

Contributor

richmeyers commented Nov 19, 2011

What happens when you have foo bar and foo+bar in the same directory?

Also, what is in the directory listings for foo'bar and foo"bar?

On Nov 19, 2011 10:59 AM, "Rich Meyers" <
reply@reply.github.com>
wrote:

What happens when you have foo bar and foo+bar in the same directory?
I did not try that.


Reply to this email directly or view it on GitHub:
#265 (comment)

Owner

raggi commented Dec 5, 2011

I am patching rack to correct the output of the Directory Index from Rack::Directory.

I will not support not escaping file names at this time. Escaping is expected and required by rack in order to conform to RFC.

If you believe this is in error, feel free to continue the discussion. At this time, supporting reading files from uris that are not correctly escaped will not be supported.

Thank you for your bug report, even though we may not agree on the solution, I have fixed the bug that is apparent in Rack::Directory.

Member

jeremy commented Feb 17, 2012

I will not support not escaping file names at this time. Escaping is expected and required by rack in order to conform to RFC.

Of course file names should be escaped. The bug is that valid URI paths are incorrectly unescaped.

For example, + is explicitly allowed in paths.

From RFC 3986:

      path          = path-abempty    ; begins with "/" or is empty
                    / path-absolute   ; begins with "/" but not "//"
                    / path-noscheme   ; begins with a non-colon segment
                    / path-rootless   ; begins with a segment
                    / path-empty      ; zero characters

      path-abempty  = *( "/" segment )
      path-absolute = "/" [ segment-nz *( "/" segment ) ]
      path-noscheme = segment-nz-nc *( "/" segment )
      path-rootless = segment-nz *( "/" segment )
      path-empty    = 0<pchar>

      segment       = *pchar
      segment-nz    = 1*pchar
      segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" )
                    ; non-zero-length segment without any colon ":"

      pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"

Where sub-delims is "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

See also http://bugs.ruby-lang.org/issues/1680

Member

jeremy commented Aug 27, 2014

This is still a bug. Worse, it's been worked around in Rails in an additionally buggy way, to great confusion and gnashing of teeth: rails/rails#11816 (comment)

The proposed fix here (removing unescaping) is wrong. The right fix is to unescape as pchar instead of the blanket Utils.unescape as a query component.

@spastorino spastorino reopened this Sep 1, 2014

@spastorino spastorino added this to the Rack 1.6 milestone Sep 1, 2014

Owner

raggi commented Nov 27, 2014

This is fundamentally the CGI heritage, which we should drop, but if we drop it, we have to do that as a major change, as we should drop it across the board, both in escape and unescape.

@spastorino spastorino modified the milestones: Rack 2.0, Rack 1.6 Dec 13, 2014

@tenderlove tenderlove added a commit that referenced this pull request Sep 4, 2015

@tenderlove tenderlove fixing support for directories that have + in the name
directories that have + in the name should be served up if the browser
puts a + in the path.  + is a valid path character, so it should not be
translated to a space (like you do in query parameters).

references #265 rails/rails#11816
978eb9b

@tenderlove tenderlove closed this in 568cf72 Sep 4, 2015

Member

jeremy commented Sep 4, 2015

giphy

@yous yous referenced this pull request Dec 9, 2016

Open

Escape URL to render HTML #1131

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment