Escaping of delimeter characters #243

Open
wants to merge 1 commit into from

4 participants

@kjbkjb

Bartosz- The attached test exhibits the behavior that I'm encountering. Thanks for looking at this. Our team really appreciates the webmock gem.

#240

@kjbkjb

Update- I've pulled down the addressable gem source and looked through their test specs. It appears as though the library appropriately parses query strings with delimiter characters. It is only when using normalize_uri that these items get escaped. It could also be the heuristic parsing since the test suite tests the method "parse".

https://github.com/sporkmonger/addressable/blob/master/spec/addressable/uri_spec.rb#L3773

I wonder if it is appropriate to use the normalized uri to make the request on behalf of your consumers. I assert that you should use the exact string that was requested. If you modify the query in any way, you are actually changing the test.

@bblimke
Owner

Do you experience webmock actually modifying the url used for real requests? Normalized uri's are just used for lookup in Webmock stub registry.

@kjbkjb

Based on the fact that VCR is reporting that the url that is being requested is not able to be handled, I'm assuming that the encoded url is used to forward the request.

@bblimke
Owner

ok, VCR uses urls provided in request callbacks. these url's are normalized indeed.

@myronmarston
Collaborator

I think there are other things that are normalized when they are provided to VCR as well--such as the request and response headers. I've occasionally run into problems with this; for example when using Fog to make requests to Amazon S3, it looks for an ETag header, but when I've used WebMock and VCR for tests involving this, the header has been Etag (due to WebMock's normalization) leading to a test that fails even though the real thing succeeds.

Ideally, I'd like to be able to get the raw, unnormalized data, because the data can't always losslessly roundtrip through the normalization process. What do you think about providing a raw method on request and response that would give me an object that has the raw, unnormalized data? It could solve this problem and the Fog ETag header problem I've experienced.

@bblimke
Owner

@myronmarston I think there should be some "rack" style interface for http clients. This interface could be utilised by WebMock or VCR or other libraries. Each http client developer team could then ensure their adapter is up to date.
WebMock was not really planned for being a unified interface across all http clients, but just be a library for mocking.

That was just an idea, but to answer your question, I'm not sure what should be the structure of the raw object. All http clients keep the data (url, params, headers, body) in different formats. Webmock normalizes them to have a unified version. How should this "unified raw structure" look like.

@myronmarston
Collaborator

I think there should be some "rack" style interface for http clients.

...which is basically what faraday is :).

All http clients keep the data (url, params, headers, body) in different formats. Webmock normalizes them to have a unified version.

I actually really like that WebMock provides one unified request and response API regardless of the HTTP client, and I would want to keep this. (It would be a nightmare for VCR to have to deal with each HTTP client differently). What I'm concerned about is the lossless normalization of the data. Normalization of data is fine as long as we can reliably roundtrip it to the original form for playback. The problem comes with a case like Etag and ETag -- both get normalized to Etag, but there's no way for VCR to know which was the original casing. Information has been lost and it can't be recovered.

Ideally, the raw structure would have an identical API to the normalized structure (maybe even using the same classes for them!), but would have raw, unnormalized headers, URL, body, etc. (I actually don't think webmock normalizes the body, thankfully, but you get the point).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment