Correct handling of "/:name.?:format?" and "/:user@?:host?" #492

Merged
merged 16 commits into from Jul 12, 2012

4 participants

@floere

This pull request adds the correct handling of the following URL path patterns to Sinatra. These are the patterns which this is about:

 Pattern             | Old Regexp                                               | Example                          | Should Be
 "/:name.?:format?"  | /^\/([^\/?#]+)(?:\.|%2E)?([^\/?#]+)?$/                   | "/foo.bar"                       | ["foo", "bar"]
 "/:name.?:format?"  | /^\/([^\/?#]+)(?:\.|%2E)?([^\/?#]+)?$/                   | "/foo%2Ebar"                     | ["foo", "bar"]                  
 "/:user@?:host?"    | /^\/([^\/?#]+)(?:@|%40)?([^\/?#]+)?$/                    | "/foo@bar"                       | ["foo", "bar"]                    
 "/:user@?:host?"    | /^\/([^\/?#]+)(?:@|%40)?([^\/?#]+)?$/                    | "/foo.foo@bar"                   | ["foo.foo", "bar"]                
 "/:user@?:host?"    | /^\/([^\/?#]+)(?:@|%40)?([^\/?#]+)?$/                    | "/foo@bar.bar"                   | ["foo", "bar.bar"]

Why? / History

This is the result of a long discussion starting here:
https://twitter.com/#!/konstantinhaase/status/182480862326165504

The discussion being here:
https://gist.github.com/2154980

What? / Code changes

I mainly touched the base.rb file, specifically the Base#compile method. My plan was to generalize some of the other cases to keep the needed changes to the code low. This plan resulted in 2 other patterns -> regexps being changed (and imho improved).

I added a quite comprehensive spec in the file compile_test.rb, which can be run with ruby test/compile_test.rb.

Possible discussion worthy points

Using /^...$/ instead of /\A...\z/ regexps

One bigger change is found in switching from /^...$/ to /\A...\z/, i.e. switching from line-based matching to global matching with anchoring at the beginning of the string. Two reasons:

  1. I believe it to be "more correct".
  2. It is thought to be more performant (for some cases, see http://www.ruby-doc.org/core-1.9.3/Regexp.html).

If this is a problem, please discuss.

Pattern "/:name.?:format?" question

I was wondering about one pattern (see header) – on an example of "/.bar" is supposed to result in [".bar", nil]. This seems strange to me – it seems to me as if it should result in a non-match, i.e. nil.
I changed the tests to reflect my line of thought. If this is untrue, please tell me why and I'll change the tests and add the commit to this pull request.

Thanks for reading! If there are any cleanups to be done, please let me know.

@rkh rkh commented on the diff Mar 23, 2012
lib/sinatra/base.rb
@@ -1302,17 +1302,21 @@ def compile!(verb, path, block, options = {})
def compile(path)
keys = []
if path.respond_to? :to_str
- pattern = path.to_str.gsub(/[^\?\%\\\/\:\*\w]/) { |c| encoded(c) }
+ ignore = ""
+ pattern = path.to_str.gsub(/[^\?\%\\\/\:\*\w]/) do |c|
+ ignore << escaped(c).join if c.match(/[\.@]/)
@rkh
Sinatra member
rkh added a note Mar 23, 2012

I think we might also want this for other symbols, like :, so maybe remove the if or something?

@floere
floere added a note Mar 23, 2012

Sure, I can do that. Do you have more example patterns -> regexps?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@rkh
Sinatra member

With this patch, does "/:foo" still parse "/foo.bar"?

@floere

(That is my answer :) )

@floere

As a note, the tests basically contain the list here: https://gist.github.com/2154980#gistcomment-168954 (with some additions)

The case "/:foo" parses "/foo.bar" wasn't in there which is why I added it.

@floere

How are we going to proceed with this? Are there any stopping points from merging? If yes, what are they?

@rkh
Sinatra member

Sorry, it's mainly a time management thing, I'd like to go over the code, again, refactor the tests, play around with it, run it in some real apps, and such. I want this to be part of the 1.4.0 release, as it is a behavioral change, but I'm also planning to have a 1.3.3 release first.

@ests

+1, this is the functionality I would love to see merged too. Consuming API webservice I've written in Sinatra, by using AcitveResource was a bit of pain, because by default AR generates request URIs like "/user/1.json" and ect.

@rkh
Sinatra member

This will be merged in for 1.4.0. I will take care of this during or after RailsConf.

@floere

@rkh Thanks for the info. If you need changes (i.e. how do the test need to be refactored?), don't hesitate to tell me.

@rkh
Sinatra member

I don't like the giant_array.each { ... } approach and would rather have some minimal DSL there.

@floere

I'm surprised. What's the reasoning? Just personal style?

@rkh
Sinatra member

Readability, easier to grab what's going on for someone new joining the project. You actually like those arrays nested in arrays nested in arrays with a bunch of code right below it?

@floere

Let's move this in a constructive direction.

DSL example:

pattern_converts_into "/:name.?:format?", /^\/([^\/?#]+)(?:\.|%2E)?([^\/?#]+)?$/
pattern_resolves "/:name.?:format?", "/foo.bar", ["foo", "bar"]

Let's go from here.

@rkh
Sinatra member

That's better, but I was thinking of maybe actually generating routes and testing if a requests goes through and what params looks like. That's the only way to actually make sure it works.

Something like this:

parses_pattern("/:name.?:format?", "/foo.bar", "name" => "foo", "format" => "bar")
does_not_parse("/:name.:format", "/")

Not sure about the method names.

@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@floere

Undecided? ;)

@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh rkh closed this Apr 19, 2012
@rkh rkh reopened this Apr 19, 2012
@rkh
Sinatra member

Ah, sorry, this was not related to the Pull Request. We tried to figure out why github is not sending out pubsub notifications for some repos/pull requests.

@floere

No worries.

@travisbot

This pull request fails (merged dfa8409 into b882ab3).

@floere

Ok. This pull request seems to be 1.9.2 only. Shall I have a look?

@rkh
Sinatra member

That'd be great. Ignore the Puma failure (fixed in master).

@travisbot

This pull request fails (merged 540b171 into b882ab3).

@floere

@travisbot Not sure who fails here ;)

Error: #<NativeException: org.virtualbox_4_1.VBoxException:
The function "powerDown" returned an error condition:
"The virtual machine is being powered down"  (0x80bb0002)>
@travisbot

This pull request fails (merged be59b2b into b882ab3).

@travisbot

This pull request passes (merged 9f08499 into b882ab3).

@travisbot

This pull request fails (merged 98ef21b into b882ab3).

@travisbot

This pull request fails (merged 57fc08d into b882ab3).

@floere

@travisbot Well, it "fails": SIGSEGV (0xb) at pc=0xb19c633b, pid=2081, tid=3078609776

Anyway, @rkh, I believe we're good to go afaics.

@travisbot

This pull request passes (merged 57babc5 into b882ab3).

@travisbot

This pull request fails (merged 6462a00 into b882ab3).

@ests

Having notifications turned on is no longer nice, got spammed by Mr. Travisbot ;)

@travisbot

This pull request fails (merged de21260 into b882ab3).

@floere

@ests Heh, I agree. See below "Disable notifications for this Pull Request".

@floere

Also, the order of comments is out of sync. The one that passes (3rd last) is the latest one.

@rkh
Sinatra member

Sorry, we are looking into optimizing @travisbot comments.

@floere

Eh, no worries. I wonder though why they can be that much out of order. Isn't some sort of queue used?

@rkh
Sinatra member

Not sure. We have them in RabbitMQ. We had some issues with Heroku and other services being down a lot the last few days, maybe something went wrong there.

@floere

Even the it's strange it's out of order, don't you think? OTOH, it's not that important (to me).

@floere

Anything stopping this one from being pulled?

@rkh
Sinatra member

Me spending too much time at conferences. I'll look into this this week, potentially today.

@floere

Hehe, thanks for the quick response. I was mainly wondering whether I can remove this from my TODO list.

@floere

Still spending too much time at conferences?

The changes (https://github.com/sinatra/sinatra/pull/492/files) are pretty small – and beautify the tests quite a bit. If I can help with the review, let me know.

@travisbot

This pull request fails (merged e59b62e into b882ab3).

@floere

Again, it doesn't really fail – it segfaults (ie. we don't know whether it would fail).

@rkh rkh merged commit 762967f into sinatra:master Jul 12, 2012
@rkh
Sinatra member

Thanks for all the effort. :)

@floere

My pleasure. Note that since the pattern behaviour changes, some apps might break who have misused the patterns that now work correctly. (I did not know where to describe that though, and forgot to ask, I'm afraid)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment