fix invalid characters in multipart uploads #515

Merged
merged 1 commit into from Apr 20, 2013

Projects

None yet

6 participants

@mraidel
Contributor
mraidel commented Feb 11, 2013

Fixes invalid characters in multipart uploads by removing all characters which are not valid. Otherwise uploads with invalid characters raise an exception (ArgumentError: invalid byte sequence in UTF-8)

@mraidel
Contributor
mraidel commented Feb 11, 2013

travis build fails because of missing bacon gem for rubinius so it has nothing to do with the pull request

@oscardelben oscardelben and 2 others commented on an outdated diff Feb 11, 2013
lib/rack/multipart/parser.rb
@@ -137,6 +137,9 @@ def get_filename(head)
if filename && filename.scan(/%.?.?/).all? { |s| s =~ /%[0-9a-fA-F]{2}/ }
filename = Utils.unescape(filename)
end
+ if filename && String.instance_methods.include?(:valid_encoding?)
@oscardelben
oscardelben Feb 11, 2013 Contributor

You can use respond_to, but can you comment on what happens on versions of Ruby prior to 1.9?

@mraidel
mraidel Feb 12, 2013 Contributor

on ruby 1.8.7 this does nothing which is fine because the problem only exists since multibyte support in ruby 1.9

@rkh
rkh Feb 12, 2013 Member

You can use respond_to

or method_defined?

@mraidel
mraidel Feb 12, 2013 Contributor

yes, method_defined? sounds good. respond_to? would work here too but it doesn't feel right to me because we would test on another instance of string, not the one we are using the method on (as we are using chars which returns new instances of String). Have pushed a fix.

@etehtsea
etehtsea commented Apr 3, 2013

What is the problem with merging this PR?

@rkh
Member
rkh commented Apr 3, 2013

No one reviewed it yet.

@jeremyevans
Contributor

I had the same problem on one of my sites. I tested this commit and it fixes the error. However, I think the implementation is suboptimal from a performance standpoint. It should only do the work of filtering the string if it is not already in a valid encoding, since checking a string for a valid encoding is very fast, rebuilding is slow, and the vast majority of filenames are already in a valid encoding. To the new if statement, you should append && !filename.valid_encoding?. This should be be about 2.5x faster for a single byte filename, 7x faster for 10 byte filename, 35x faster for a 100 byte filename, and close to 300x faster for a 1KB filename.

With that change I think this should go in.

@spastorino
Member

👍 to what @jeremyevans said. So @mraidel can you do that modification and squash your commits?

@spastorino
Member

And also your test https://travis-ci.org/rack/rack/jobs/4745776 fail in Ruby 2.0

@spastorino
Member

@mraidel You will need to properly encode the String in the test

@mraidel
Contributor
mraidel commented Apr 20, 2013

@jeremyevans @spastorino I have included both recommended changes and squashed the commits!

@spastorino
Member

@raggi @rkh 👍 good to merge

@rkh rkh commented on the diff Apr 20, 2013
lib/rack/multipart/parser.rb
@@ -137,6 +137,9 @@ def get_filename(head)
if filename && filename.scan(/%.?.?/).all? { |s| s =~ /%[0-9a-fA-F]{2}/ }
filename = Utils.unescape(filename)
end
+ if filename && String.method_defined?(:valid_encoding?) && !filename.valid_encoding?
+ filename = filename.chars.select { |char| char.valid_encoding? }.join
+ end
@rkh
rkh Apr 20, 2013 Member

On 2.1, we could use String#scrub, btw.

@rkh rkh merged commit a811f12 into rack:master Apr 20, 2013

1 check passed

default The Travis build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment