Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

query parameters are erroneously unescaped #226

Closed
FND opened this Issue · 10 comments

4 participants

@FND

minimal test case:

require "net/http"
require "cgi"
require "webmock"

WebMock.stub_request(:any, /.*/).
    with { |req| puts "AFTER : #{req.uri.query}"; true }

uri = URI.parse("http://example.org/?q=#{CGI.escape("foo/bar")}")
puts "BEFORE: #{uri.query}"
req = Net::HTTP::Get.new(uri.to_s)
Net::HTTP.new(uri.host, uri.port).request(req)

expected:

BEFORE: q=foo%2Fbar
AFTER : q=foo%2Fbar

actual:

BEFORE: q=foo%2Fbar
AFTER : q=foo/bar

I assume this is not intentional?

@der-flo

I have a similar problem. My escaped query params contain non-UTF-8 characters, so webmock fails here:

ArgumentError: invalid byte sequence in UTF-8
  occurred at /usr/local/lib/ruby/gems/1.9.1/gems/webmock-1.11.0/lib/webmock/util/uri.rb:91:in `gsub'
@bblimke
Owner

@der-flo can you please provide an example of a code that fails?

@der-flo

@bblimke sure. I took @FND's snippet as base:

# encoding: utf-8

require 'net/http'
require 'cgi'
require 'webmock'

WebMock.stub_request(:any, /.*/).with { |r| puts r.uri.query; true }

param = 'aäoöuü'.encode('iso-8859-1')
param = CGI.escape(param)

uri = URI.parse("http://www.example.org/?#{param}")
puts uri.query
req = Net::HTTP::Get.new(uri.to_s)
Net::HTTP.new(uri.host, uri.port).request(req)

This results in the following exception:

[...]/gems/webmock-1.11.0/lib/webmock/util/query_mapper.rb:83:in `block in query_to_values': invalid byte sequence in UTF-8 (ArgumentError)
    from [...]/gems/webmock-1.11.0/lib/webmock/util/query_mapper.rb:63:in `each'
    from [...]/gems/webmock-1.11.0/lib/webmock/util/query_mapper.rb:63:in `inject'
    from [...]gems/webmock-1.11.0/lib/webmock/util/query_mapper.rb:63:in `query_to_values'
    from [...]gems/webmock-1.11.0/lib/webmock/util/uri.rb:17:in `block in <class:URI>'
    from [...]/gems/webmock-1.11.0/lib/webmock/util/uri.rb:33:in `yield'
    from [...]/gems/webmock-1.11.0/lib/webmock/util/uri.rb:33:in `default'
    from [...]/gems/webmock-1.11.0/lib/webmock/util/uri.rb:33:in `normalize_uri'
    from [...]/gems/webmock-1.11.0/lib/webmock/request_signature.rb:10:in `initialize'
    from [...]/gems/webmock-1.11.0/lib/webmock/http_lib_adapters/net_http.rb:266:in `new'
    from [...]/gems/webmock-1.11.0/lib/webmock/http_lib_adapters/net_http.rb:266:in `request_signature_from_request'
    from [...]/gems/webmock-1.11.0/lib/webmock/http_lib_adapters/net_http.rb:75:in `request'
    from main.rb:15:in `<main>'

I hope this helps.

@bblimke
Owner

@der-flo thanks. your issue is different so I suggest you create a separate issue in github.

@bblimke
Owner

@FND if this issue is still relevant, what output would you expect? WebMock doesn't modify the actual request uri,
it's just how the uri string is internally represented in WebMock (unescaped).

@FND

I'm not currently using WebMock, but would still consider this relevant.

As illustrated in the OP (though perhaps not sufficiently clear), I would expect the with block's req.uri.query to not be decoded, i.e. I would want it to match the original incoming data.

@bblimke
Owner

@FND I see. I will have to think about it as it's not that trivial. Some http clients allow you to specify query params using hash in addition to url string. WebMock tries to keep the unified form of the url across all http clients.

@FND

I appreciate that. It's probably a bit of a niche issue - but pretty puzzling when you encounter it.

FWIW, I originally tripped over this issue in innoq/iq_triplestorage@b358275#L2R33 - skimming it now, that test seems a bit obscure, but I'm insufficiently contextualized to decide whether that was a good idea in the first place.

@presidentJFK
Collaborator

closing this as it looks like it was solved by #308. If it different than that issue, please reopen. Thanks for submitting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.