RFC 6570 (URI Template) support for URIPattern #334

merged 7 commits into from Mar 5, 2014


None yet

3 participants

maxlinc commented Nov 20, 2013

This is my attempt at adding RFC 6570 support to WebMock. WebMock was already using Addressable::URI, this just hooks it into Addressable::Template as well.

I am interested in this because many services I want to mock are using (or starting to use) RFC 6570 URI Templates in their documentation. This let's me copy a URL out of the documentation and straight into WebMock. For example, OpenStack documentation is using RFC 6570, as does any documentation using apiary.io or apiblueprint. GitHub's API docs don't use it, but their API returns RFC 6570 links in the hypermedia section, just like json-hyper-schema.

I know there's lots of other URI matching systems out there, but I believe the maintenance cost of this would be lower than others, since it is a standard and is implemented by an existing WebMock dependency.

I did come across two issues, though:

  • Addressable::Template only seems to support strict matching. If an unknown query parameter is received it will not match the pattern.
  • It is tricky to use Addressable::Template along with WebMock's .with(:query => ...), since the former matches full URLs and the latter matches only a portion.

My original implementation raised and exception if you combined Addressable::Template with query_params set via WebMock. I then switched it so that Addressable::Template matches the full URI if no query_params are passed to WebMock. If a query_param is passed, then a warning is displayed and Addressable::Template is used to match the base uri, and WebMock is used to match the query params.

maxlinc commented Nov 21, 2013

Hmm... everything passed except jruby-19mode, which looks like it failed in an unrelated test. I installed jruby-1.7.8 locally but couldn't reproduce the problem.

The failing test was:
"HTTPClient when using async requests with WebMock when request stub declares that request should return a response when sequences of responses are declared should return responses one by one if declared with several to_return invokations"

So perhaps "async requests" plus "sequences of responses" causes a race condition?

bblimke commented Nov 22, 2013

@maxlinc That's really cool! I wonder if rfc6570 templates are rich enough to be used internally inside WebMock,
as a replacement for current URLPatten.

One thing I don't like is creating additional dependency on Addressable If it's worth adding rfc 6570 to Webmock, I'm not planning to implement it from scratch though.
I'm not very familiar with rfc 6570 and would like to read more about it before merging. Is rfc doc best place or do you know a better tutorial?

maxlinc commented Nov 22, 2013

Glad you're intrigued. This would be really useful for me, and I think others as well. If you have more questions or concerns let me know and I'll try to get answers or find someone who can.

The RFC isn't bad (for an RFC). In particular, section section 1.2: Levels and Expression Types seems like the meat of the RFC. There are also many implementations where you'll find basic guides: Addressable, uri_template and mustermann for Ruby, plus non-Ruby implementations. Mustermann is powerful and interesting, but it requires Ruby 2+.

One interesting thing to note about RFC 6570 is that it can work in both ways. You can pass variables to a template to "expand" it and produce a valid URL. You can also take a URL and match it to a template to "extract" URI params into a hash. The latter is useful for me in WebMock after_request callbacks, though I have the RequestSignature available so I need some extra layers to get from RequestSignature back to URITemplate.

I don't think it could be a full replacement for URIPattern, but it is probably subject to the 80/20 rule. I think the majority of users could use RFC 6570, especially if they are using WebMock to test RESTful services. However, the RFC does warn that for advanced patterns you may still need regular expressions:

Some URI Templates can be used in reverse for the purpose of variable
matching: comparing the template to a fully formed URI in order to
extract the variable parts from that URI and assign them to the named
variables. Variable matching only works well if the template
expressions are delimited by the beginning or end of the URI or by
characters that cannot be part of the expansion, such as reserved
characters surrounding a simple string expression. In general,
regular expression languages are better suited for variable matching.

Internally, Addressable and other implementations seems to use regexes, so in some ways uri-templates are short-hand regexes. Unfortunately they don't have a template.to_regex.

bblimke commented Nov 24, 2013

Thanks. It would be good to add some info to the readme. Except that it's good to merge.

maxlinc commented Nov 25, 2013

Great, thanks! I'll get to that soon - probably tomorrow.

maxlinc commented Mar 4, 2014

Updated the readme a while back. Forgot to add a comment to let you know. Travis is green, except that one non-deterministic async test, which failed on ree only.

Need anything else?

@bblimke bblimke merged commit 0f0de80 into bblimke:master Mar 5, 2014

1 check failed

default The Travis CI build failed
bblimke commented Mar 5, 2014

Thanks for update.

maxlinc commented Mar 10, 2014

FYI - I think sporkmonger/addressable#140 fixes both of the issues I mentioned in the original description.

@maxlinc maxlinc deleted the maxlinc:addressable branch Mar 10, 2014
@maxlinc maxlinc referenced this pull request in sporkmonger/addressable Mar 10, 2014

Fixes for #137 and #131 #140


@bblimke At this point, the biggest reason not to use an Addressable dependency is that almost everything else already has one. Recipe for dependency conflicts.

However, Addressable is about as lightweight and stable as a gem will ever be. Aside from one change to the query_values method and an update to implement the final version of RFC 6570, very little has changed in Addressable in the past 3 years. It'd be pretty safe to set a fairly permissive '>= 2.3.6' dependency rule (2.3.6 would be the release that has the fixes you need, though I haven't cut the release yet). When I do finally get around to releasing 3.0, I expect a few minor breaking changes to anyone who's subclassing the URI class, but that's about it and it'll affect a very tiny minority. Most people will never benefit from ~> paranoia with Addressable.


FYI, Addressable 2.3.6 has been released.

bblimke commented Mar 24, 2014

@sporkmonger thanks. I'll update gemspec to ">= 2.3.6"


I did find an obscure bug immediately after cutting the release but it shouldn't affect most real-world scenarios. If you use libidn+idn-ruby, a host of "example..com" (two dots) will give an IDNA error on normalize. Bug already fixed in master, but I probably won't do an instant follow up release unless people actually start hitting that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment