query string route syntax & parameter-based URI decoding #668

Closed
wants to merge 18 commits into
from
@jhudson8

Query string route syntax

Any route except those ending with a wildcard will automatically accept additional content using the '?' separator. This content is a set of key value pairs using '&' as the pair separator and '=' as the key/value separator - just like the URL query string syntax.

If a query string exists in the route hash, the routing function (defined in the routes hash) will be given an additional parameter (in last ordinal position) containing hash of the key/value data.

routes: {
'foo/:bar': 'myRoute'
}
...
myRoute: function(bar, params) {
// the params attribute will be undefined unless there is a route containing a query string
}

Example route patterns

#foo/abc -> myRoute( 'abc', undefined )

#foo/abc?foo=bar -> myRoute( 'abc', {'foo': 'bar'} )

Nested query string Keys

Any keys containing '.' will represent a nested structure.

#foo/abc?me.fname=Joe&me.lname=Hudson -> myRoute('abc', {'me': {'fname': 'Joe', 'lname': 'Hudson'}} )

Query string value arrays

Any values containing '|' will assume an array structure. Note: you can still have non-array values containing '|' but it must be URI encoded (%7C).

You can prefix the value with '|' to ensure an array in case there is only a single value.

#foo/abc?animals=cat|dog -> myRoute( 'abc', ['cat', 'dog'] )

#foo/abc?animals=|cat -> myRoute( 'abc', ['cat'] )

Generating route with query string=========================

You can easily create a route string from a route + parameter hash using the router.toFragment(route, parameters) method. It can contain a nested hash structure or arrays. ex:
var route = router.toFragment('myroute', {
a:'l', b:{c: 'n', d:'m', e:{f: 'o'}}, array1:['p'], array2:['q', 'r'], array3:['s','t','|']
});

Parameter-based URI decoding

Current backbone will decode the complete hash value. This requires route parameters to be encoded multiple times if they contain reserved characters ('/' for example).

This patch moves all decoding to route parameters when they are in parameter form (after the hash has been parsed).

jhudson8 added some commits Oct 14, 2011
@jhudson8 jhudson8 - changed hash decoding to individual parameters (so parameter data d…
…oes not have to be encoded twice if it contains route reserved characters)

- added new route syntax routeName/?
  - will match route 'routeName/a=b&c=d'
  - can still use route parameters -> routeName/:foo/?
  - the last route controller function argument is a hash containing key/values matching the query string (with values URI decoded)
9d9e5ec
@jhudson8 jhudson8 fixed query string route test bug f4798d5
@jhudson8 jhudson8 added query string and URI decoding tests 36afe19
@ricardovf

Nice!

Would be nice to have an method to set/update the query string on the current URL.

@lapidus

+1

jhudson8 added some commits Oct 25, 2011
@jhudson8 jhudson8 change credit to Kevin Decker
This allows for routes that do not have a / separating the param section and the path section, i.e.

        "ip/:name/:id?": "item",
        "ip/:id?": "item",
34da7a9
@jhudson8 jhudson8 added test case and fixed bug 30779b8
@jhudson8 jhudson8 no longer need to explicitely define the ? in the route definition fo…
…r query parameters.

any route ending with ?... will get an aditional hash with the parameters
3a2cf52
@jhudson8 jhudson8 removed console log statements b2ad52c
@jhudson8 jhudson8 added reserved char test case that was removed a01c507
@wookiehangover
Collaborator

maybe a good call to add some updated documentation to go along with this?

@lapidus

Does it also decode objects encoded in the query string? I.e. "?a[b]=c"
(I tried, but only seemed to get "primitive types".)

@eastridge

Joe, I believe we discussed this in Campfire but ++ decoding nested objects in the query string.

@jhudson8

It doesn't currently but can be done - I'll see if I can get that in there

@lapidus

Nice. I've been using this in the past:
http://benalman.com/code/projects/jquery-bbq/examples/deparam/

Something along these lines:

routeHandler: function(params) {
var paramsObject = jQuery.deparam( params );
}

jhudson8 added some commits Nov 2, 2011
@jhudson8 jhudson8 added complex key structure support
ex: #foo?foo[a]=b

  {
    'foo': {
      'a': 'b'
    }
  }
dd43d38
@jhudson8 jhudson8 changed hash notation to a.b.c
added value array notation using a|b|c
625d27b
@jhudson8 jhudson8 fixed ios4 specific bug with splat routes 22a52a8
@jhudson8 jhudson8 added includeQueryPath parameter to getFragment
removed query string content from getFragment by default
added getQueryString method
f13d06b
@jhudson8 jhudson8 added fragment with/without query string tests 9cdc592
@jhudson8 jhudson8 use array structure for values if multiple query parameters of the sa…
…me name are encountered
90884ba
@jhudson8 jhudson8 getFragment does not default to removing the query string (but is sti…
…ll available as a method parameter) - so it works just like it used to
41879f4
@jhudson8 jhudson8 added router toFragment method to serialize a hash/array structure in…
…to the appropriate route with query string parameters
0b25ab9
@jhudson8

Question to anyone participating in this pull request:

Would you rather have a helper method for creating a url with parameters
1) directly on Backbone.history.navigate - more convienant to use
2) in the router - in the same place that parameter decoding is
3) not at all

I've just added it to the router but want to get other opinions... Thanks.

@ricardovf

jhudson8, great work. I think it should be a method on the Router, so i can use it stand alone and for other uses, like to put it in href of some a tags and not really change the url to it.

@eastridge

Great work Joe. Since navigate already has extra arguments on it I'd say keep it separate. The example you posts with the encoding done in toFragment seems like it would suffice.

Is there a reason you went with "." as the object separator in the query string? I'd prefer to see:

"address[street]" rather than "address.street" to match PHP and Rails.

@ricardovf

I agree with syntacticx.

Maybe it should be optional as dot is really cleaner and using [] will only be useful if you are going to use it with HTML5 history (not using #) and really decode the URLs on the server side.

@eastridge

@ricardovf, I'm arguing for using brackets rather than the dot syntax.

@ricardovf

@syntacticx my comment was confusing

Im arguing to make it optional to use dot or brackets since i see the point of using both depending on the project/objective. Dunno if its gonna be a hard thing thought.

@jhudson8

I previously implemented the a[b] syntax and got feedback requesting the a.b syntax. I ended up realizing that I can't satisfy everyone and went with what I prefer which is the a.b syntax. There are hooks to easily change the behavior to suit your needs.

_setParamValue - for decoding the parameters
_toQueryParamName - for encoding the parameters (specifically, creating the name for the parameter)

@jashkenas
Owner

So -- this is a fantastic feature for a Backbone plugin, and I highly encourage you to wrap it up and publish it as such on the Wiki.

But I don't think that query parameters in routes make sense for Backbone core. A big part of having a Backbone.Router, instead of using something like jQuery BBQ, is so that you have clean, pretty URL fragments like:

/search/madoff/p10

Instead of the nastier:

?action=search&query=madoff&page=10

The vast majority of Backbone apps will have a limited set of client-side URLs that can be easily expressed in the former notation. There may be some that have such a wide variety of URL possibilities that query params are a better choice, but they're going to be in the minority, and it's not worth adding this amount of code to core Backbone to help 'em out, when a plugin will work equally well.

Finally, whereas query parameters are the only way to communicate data to the server side in a GET request ... this is the client-side were talking about here, and a large number of your query params are probably better passed around in pure JavaScript, and not jammed into the URL, unless they're something you truly intend to make bookmarkable.

@jashkenas jashkenas closed this Nov 30, 2011
@jhudson8

:(

I understand - thanks for taking the time to evaluate.

I would like to mention that it was never intended as a mutually exclusive deal... #/foo/bar?a=b&c=d

The problem is that the pretty URLs become much less pretty when you have optional parameters. The other problem is that there is also a fragment decoding issue (fragment is decoded before splitting it into parts) which makes parameters that contain special characters ('/') have to be encoded more than once.

@jashkenas
Owner

Just out of curiosity, in your actual app -- what are some examples of real client-side URLs that take advantage of these query params?

@jhudson8

Jeremy, will get details for you - I'm going to make sure it's ok to tell you who I work for (I think it's fine but just in case)...

In the meantime: https://github.com/documentcloud/backbone/wiki/Adding-Query-Parameter-support

I assume this is what you mean by a plugin - please let me know if there is a better way. Thanks.

@ricardovf

@jashkenas github uses it:

https://github.com/documentcloud/backbone/issues?labels=enhancement&sort=updated&direction=desc&state=open&page=1

The route could be: /documentcloud/backbone/issues

And everytime the query changes the route would be called with diferente query strings.

As you can note for exemple in the Labels selector (bottom left) the href of the links is changed in real time with updated url everytime i use a diferente filter.

The advantage of jhudson is that is really simplier to deal with an object (from query string) then from custom uri, like: /search/madoff/p10

@jhudson8

Also, if this functionality is to remain as a plugin, I would like to request that the fragment uri decoding be pushed to _extractParameters - out of getFragment. Doing this would all this type of plugin with minimal risk to backbone core changes. Thoughts?

@jashkenas
Owner

Yep -- extracting the decoding to an overridable function sounds like a fine idea.

@jhudson8

Cool, then this really works just fine as a plugin - and you only have to get it if you want it. Thanks Jeremy.

@jashkenas
Owner

@jhudson8 -- no, a massive Wiki paste is not what I mean by a plugin. ;)

Instead, make a github project that includes a script that provides the override functionality. So if I load backbone.js, and then backbone.queryparams.js ... I'll have a working version.

@jhudson8

:) thanks

@clutchski

For what it's worth, my team has added hash string query parameters as well. URL fragments don't really scale or handle optional parameters. Also, returning a hash of unordered params from routes is nice, because adding or removing parameters doesn't require a change to the routing code.

@christo

A case for adding this functionality to core:

Whenever parameters can have a "/" in them, those values cannot be made to work in path params if tomcat or apache are involved. %2F is the URL encoded form of "/" but that value will not hide the "/" in the path param.

This means query params for any parameters whose values could possibly contain a "/", such as search queries. So search queries must go in the query params and therefore the backbone routes break down.

Slosh "\" is also impossible to work with in this way.

There is one case in which a "/" can work with path params, that is if it is the last path param and the slash is NOT URI encoded.

The relevant spec reference: http://labs.apache.org/webarch/uri/rfc/rfc3986.html#percent-encoding

"The purpose of reserved characters is to provide a set of delimiting characters that are distinguishable from other data within a URI. URIs that differ in the replacement of a reserved character with its corresponding percent-encoded octet are not equivalent. Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications. Thus, characters in the reserved set are protected from normalization and are therefore safe to be used by scheme-specific and producer-specific algorithms for delimiting data subcomponents within a URI."

@russelldavis

@jhudson8, any plans to make this into a plugin like @jashkenas suggested?

@jhudson8

russelldavis,

Thanks for reminding me. I'll do it soon.

@naid

thank you jhudson8, that helped us a lot =)

@jhudson8

No problem, I'm glad it helped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment