Reverse variable matching #2

Open
aslakhellesoy opened this Issue Jun 29, 2012 · 33 comments

Comments

Projects
None yet
@aslakhellesoy

I couldn't find any information about this in the docs or by skimming the source code. For example - given a template /foo/{name}/bar/{id} and a URI /foo/hello/bar/world - be able to extract the name and id variables with values hello and world respectively.

Would this be within the scope of this library? The wo-furi library supports this, but since that library seems abandoned and implements an older version of the spec I'm looking for something else.

FWIW, I'd like to use this in webbit-rest

@aslakhellesoy

This comment has been minimized.

Show comment Hide comment
@aslakhellesoy

aslakhellesoy Jun 29, 2012

Follow-up - the ruby addressable project has an implementation that does this: https://github.com/sporkmonger/addressable/blob/master/lib/addressable/template.rb

It essentially turns the pattern into a regexp and then matches against the uri and then loops over the capture groups to extract variable values.

Follow-up - the ruby addressable project has an implementation that does this: https://github.com/sporkmonger/addressable/blob/master/lib/addressable/template.rb

It essentially turns the pattern into a regexp and then matches against the uri and then loops over the capture groups to extract variable values.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Jun 29, 2012

Owner

If I'm understanding this correctly, you want to see if a URI matches a given expression and if it does, you want to be able extract the variable values from the URI? Obvious use case to match request URIs to patterns on the server. If that correct?

Owner

damnhandy commented Jun 29, 2012

If I'm understanding this correctly, you want to see if a URI matches a given expression and if it does, you want to be able extract the variable values from the URI? Obvious use case to match request URIs to patterns on the server. If that correct?

@aslakhellesoy

This comment has been minimized.

Show comment Hide comment
@aslakhellesoy

aslakhellesoy Jun 29, 2012

Yes, that's exactly it. As mentioned in section 1.4 in the spec:

Some URI Templates can be used in reverse for the purpose of variable
matching: comparing the template to a fully formed URI in order to
extract the variable parts from that URI and assign them to the named
variables.

Yes, that's exactly it. As mentioned in section 1.4 in the spec:

Some URI Templates can be used in reverse for the purpose of variable
matching: comparing the template to a fully formed URI in order to
extract the variable parts from that URI and assign them to the named
variables.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Jun 29, 2012

Owner

Got it. I'll target this for the 1.2 release. I have been refactoring the code a bit to separate the expression model from the template in order to make issue #1 a bit easier. This feature request fits nicely into the same refactoring. It'll take a few more days but doable.

Owner

damnhandy commented Jun 29, 2012

Got it. I'll target this for the 1.2 release. I have been refactoring the code a bit to separate the expression model from the template in order to make issue #1 a bit easier. This feature request fits nicely into the same refactoring. It'll take a few more days but doable.

@mwanji

This comment has been minimized.

Show comment Hide comment
@mwanji

mwanji Aug 16, 2012

+1 to this request. Would be extremely useful for a web framework I'm writing.

mwanji commented Aug 16, 2012

+1 to this request. Would be extremely useful for a web framework I'm writing.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Aug 20, 2012

Owner

I've been working on this here and there over the past few months and it's nearly complete. As a result of this new functionality, it will change the API considerably. For starters, the UriTemplate will now be immutable and there will be a UriTemplate.Builder class that will be used to programmatically construct a URI template. Lastly, this change will introduce a UriMatcher that will match a URI against a URI template pattern.

Owner

damnhandy commented Aug 20, 2012

I've been working on this here and there over the past few months and it's nearly complete. As a result of this new functionality, it will change the API considerably. For starters, the UriTemplate will now be immutable and there will be a UriTemplate.Builder class that will be used to programmatically construct a URI template. Lastly, this change will introduce a UriMatcher that will match a URI against a URI template pattern.

@lalloni

This comment has been minimized.

Show comment Hide comment
@lalloni

lalloni Sep 4, 2012

+1 to this request.

Has been any advance in this issue?

Is there any schedule for release?

We're needing this so bad that we'll roll our own if not available.

lalloni commented Sep 4, 2012

+1 to this request.

Has been any advance in this issue?

Is there any schedule for release?

We're needing this so bad that we'll roll our own if not available.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Sep 4, 2012

Owner

Realistically, it's 2 to 3 weeks out. Juggling summer and my day job, just getting back into the swing of things

Owner

damnhandy commented Sep 4, 2012

Realistically, it's 2 to 3 weeks out. Juggling summer and my day job, just getting back into the swing of things

@kminder

This comment has been minimized.

Show comment Hide comment
@kminder

kminder Oct 22, 2012

+1. I'm in the same boat as plalloni. Will either have to deal with wo-furi or roll our own and then hopefully replace with Handy when this feature is available.

While I'm at it the other thing we will need to layer on top is to be able to pick the "best" match for a given URI from a set of candidate URITemplates

kminder commented Oct 22, 2012

+1. I'm in the same boat as plalloni. Will either have to deal with wo-furi or roll our own and then hopefully replace with Handy when this feature is available.

While I'm at it the other thing we will need to layer on top is to be able to pick the "best" match for a given URI from a set of candidate URITemplates

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Oct 22, 2012

Owner

On the subject of reverse mapping, this is a bit more challenging that one might think. This is especially true when you start looking at reverse mapping for level 3 and level 4 expression types. level 1 and level 2 expressions are fairly straight forward and that's close to what wo-furi supports now. I've been proceeding with the idea of trying to be bale to reverse map level 4 expressions. If folks find levels 1 and 2 acceptable for the time being, that's a tad easier.

On the topic of rolling your own implementation, let's remember that this is GitHub and this is an open source project. I'm pretty liberal on contributions. I've already taken in a few bug fixes and feature contributions are more than welcome.

Owner

damnhandy commented Oct 22, 2012

On the subject of reverse mapping, this is a bit more challenging that one might think. This is especially true when you start looking at reverse mapping for level 3 and level 4 expression types. level 1 and level 2 expressions are fairly straight forward and that's close to what wo-furi supports now. I've been proceeding with the idea of trying to be bale to reverse map level 4 expressions. If folks find levels 1 and 2 acceptable for the time being, that's a tad easier.

On the topic of rolling your own implementation, let's remember that this is GitHub and this is an open source project. I'm pretty liberal on contributions. I've already taken in a few bug fixes and feature contributions are more than welcome.

@lalloni

This comment has been minimized.

Show comment Hide comment
@lalloni

lalloni May 16, 2013

Got back to this after some time. After all we just went with ugly hand made uri parsing & building.

@damnhandy I think level 1 & 2 would be great and enough for most cases.

@damnhandy WRT "roll our own": at that time, I was talking about rolling our own just because we target the scala language, not java, so if any effort would've been undertaken it would've been more gain for us if we offered a scala API

lalloni commented May 16, 2013

Got back to this after some time. After all we just went with ugly hand made uri parsing & building.

@damnhandy I think level 1 & 2 would be great and enough for most cases.

@damnhandy WRT "roll our own": at that time, I was talking about rolling our own just because we target the scala language, not java, so if any effort would've been undertaken it would've been more gain for us if we offered a scala API

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Jul 11, 2013

Owner

Same here @plalloni , been a while. I've restructured the code to better support this as the initial design presented some problems. There's going to be some API changes, but this will pave the way to reverse matching as well as much better feedback on parsing errors. Version 2.0 should be available in a few days and then we'll get reverse matching in 2.1.

Owner

damnhandy commented Jul 11, 2013

Same here @plalloni , been a while. I've restructured the code to better support this as the initial design presented some problems. There's going to be some API changes, but this will pave the way to reverse matching as well as much better feedback on parsing errors. Version 2.0 should be available in a few days and then we'll get reverse matching in 2.1.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Sep 24, 2013

Owner

Just a heads up, I'm planning on using Java 7 since the regex supports named groups, which in turn will make this process a lot easier. If anyone has any objections, please let me know. I'm also assuming that since Java 6 was EOL'd back in February, most folks have moved on from Java 6.

Owner

damnhandy commented Sep 24, 2013

Just a heads up, I'm planning on using Java 7 since the regex supports named groups, which in turn will make this process a lot easier. If anyone has any objections, please let me know. I'm also assuming that since Java 6 was EOL'd back in February, most folks have moved on from Java 6.

@henrib

This comment has been minimized.

Show comment Hide comment
@henrib

henrib Nov 21, 2013

+1 for the request; as a temporary measure, adding the URITemplateResolver and test from 2.1 branch in the 2.0 trunk using Java7 tests ok. Any status on level 3/4 resolution ?

henrib commented Nov 21, 2013

+1 for the request; as a temporary measure, adding the URITemplateResolver and test from 2.1 branch in the 2.0 trunk using Java7 tests ok. Any status on level 3/4 resolution ?

@felipesere

This comment has been minimized.

Show comment Hide comment
@felipesere

felipesere Apr 15, 2014

What is the state on this? I could not find any Class to perform matching or variable extraction :)

What is the state on this? I could not find any Class to perform matching or variable extraction :)

@felipesere

This comment has been minimized.

Show comment Hide comment
@felipesere

felipesere Aug 2, 2014

That looks pretty interesting. I'd like to go over the code and make a pull request. Should I do it against Master or the 2.1-branch?

That looks pretty interesting. I'd like to go over the code and make a pull request. Should I do it against Master or the 2.1-branch?

@faisalferoz

This comment has been minimized.

Show comment Hide comment
@faisalferoz

faisalferoz Aug 3, 2014

IMO should be against 2.1- branch. @damnhandy shud be able to tell better since he is the owner of the project.

IMO should be against 2.1- branch. @damnhandy shud be able to tell better since he is the owner of the project.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Aug 3, 2014

Owner

Yes @faisalferoz, use the 2.1 branch.

Owner

damnhandy commented Aug 3, 2014

Yes @faisalferoz, use the 2.1 branch.

@kasun04

This comment has been minimized.

Show comment Hide comment
@kasun04

kasun04 Oct 26, 2015

I think this is a very useful feature. Is this feature available in the current 2.0.4 release? If not when will be added to a GA release?

Thanks,
Kasun.

kasun04 commented Oct 26, 2015

I think this is a very useful feature. Is this feature available in the current 2.0.4 release? If not when will be added to a GA release?

Thanks,
Kasun.

@drdamour

This comment has been minimized.

Show comment Hide comment
@drdamour

drdamour Feb 23, 2016

came looking for this functionality...how close is it? is there any stepping stones? what is this regex pattern stuff?

came looking for this functionality...how close is it? is there any stepping stones? what is this regex pattern stuff?

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 May 18, 2016

Although an old issue, I think this is still a very useful and desirable feature. I'd be willing to put some time in to development if @damnhandy can point me in the right direction.

kag0 commented May 18, 2016

Although an old issue, I think this is still a very useful and desirable feature. I'd be willing to put some time in to development if @damnhandy can point me in the right direction.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Jul 3, 2016

Owner

@kag0 I could definitely use a hand on defining the API for this feature and some test cases.

When I started the 2.x release, I refactored everything so that there is a structured model that represents the URI Template. A URI template consists of a collection of URITemplateComponent instances, which has two specializations:

  • Expression: this represents the a URI template expression such as {?foo}, etc.
  • Literal which represents the bits that are not URI template expressions

An Expression is composed of an Operator and a collection of VarSpec instances. The model goes something like this:

  • UriTemplate:
    • UriTemplateComponent:
      • Literal
      • Expression:
        • Operator
        • VarSpec

What I had intended to do here is generate a regex match string for each VarSpec, which the Expression could then use build a regex for the expression, all the way up to the UriTemplate level.

Getting the regexes correct had been frustrating, mainly because I suck at it :) I'm not entirely sure the approach I'm taking is ideal, but it seemed to make sense at the time. The last tricky bit is getting the reverse matching accuracy down. In places, it's going to be tough to distinguish which URI matches given template given how the regex is constructed.

Owner

damnhandy commented Jul 3, 2016

@kag0 I could definitely use a hand on defining the API for this feature and some test cases.

When I started the 2.x release, I refactored everything so that there is a structured model that represents the URI Template. A URI template consists of a collection of URITemplateComponent instances, which has two specializations:

  • Expression: this represents the a URI template expression such as {?foo}, etc.
  • Literal which represents the bits that are not URI template expressions

An Expression is composed of an Operator and a collection of VarSpec instances. The model goes something like this:

  • UriTemplate:
    • UriTemplateComponent:
      • Literal
      • Expression:
        • Operator
        • VarSpec

What I had intended to do here is generate a regex match string for each VarSpec, which the Expression could then use build a regex for the expression, all the way up to the UriTemplate level.

Getting the regexes correct had been frustrating, mainly because I suck at it :) I'm not entirely sure the approach I'm taking is ideal, but it seemed to make sense at the time. The last tricky bit is getting the reverse matching accuracy down. In places, it's going to be tough to distinguish which URI matches given template given how the regex is constructed.

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 Jul 3, 2016

@damnhandy that seems like a reasonable way to go about it. I'll start playing around with it. Is master the best branch to look at as far as reverse matching changes?

kag0 commented Jul 3, 2016

@damnhandy that seems like a reasonable way to go about it. I'll start playing around with it. Is master the best branch to look at as far as reverse matching changes?

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Jul 3, 2016

Owner

Use the 2.2-reverse-matching branch. Let's keep it out of master for now. The work I had done to date on this feature had led to other bugs, so I'd rather keep this in a separate branch for the time being.

Owner

damnhandy commented Jul 3, 2016

Use the 2.2-reverse-matching branch. Let's keep it out of master for now. The work I had done to date on this feature had led to other bugs, so I'd rather keep this in a separate branch for the time being.

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 Jul 4, 2016

@damnhandy I've taken a quick 'n dirty stab at it here and jotted some notes. Check it out if you think you might have some comment, otherwise I'll keep working at it.

kag0 commented Jul 4, 2016

@damnhandy I've taken a quick 'n dirty stab at it here and jotted some notes. Check it out if you think you might have some comment, otherwise I'll keep working at it.

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Aug 3, 2016

Owner

This looks good. Can you issue a PR to the 2.2-reverse-matching branch? Next step is to figure out how to integrate this into the project. The furi project that @aslakhellesoy mentioned has a dedicated URIResolver class to perform the reverse lookup. We could do something similar, but I'm not sure about the multiple matching rules approach. An suggestions are appreciated.

Owner

damnhandy commented Aug 3, 2016

This looks good. Can you issue a PR to the 2.2-reverse-matching branch? Next step is to figure out how to integrate this into the project. The furi project that @aslakhellesoy mentioned has a dedicated URIResolver class to perform the reverse lookup. We could do something similar, but I'm not sure about the multiple matching rules approach. An suggestions are appreciated.

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 Aug 15, 2016

@damnhandy checked out furi. Could be something there, but might be tricky given how the structures of the two libraries are different. In the meantime I went ahead and finished the feature more or less.

kag0 commented Aug 15, 2016

@damnhandy checked out furi. Could be something there, but might be tricky given how the structures of the two libraries are different. In the meantime I went ahead and finished the feature more or less.

@brianm

This comment has been minimized.

Show comment Hide comment
@brianm

brianm Sep 1, 2016

Awesome! /me is eagerly anticipating 2.2 now :-)

brianm commented Sep 1, 2016

Awesome! /me is eagerly anticipating 2.2 now :-)

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 Dec 17, 2016

Hey @damnhandy, just checkin in. Is there anything that could be done to help speed up a 2.2 release?

kag0 commented Dec 17, 2016

Hey @damnhandy, just checkin in. Is there anything that could be done to help speed up a 2.2 release?

@damnhandy

This comment has been minimized.

Show comment Hide comment
@damnhandy

damnhandy Dec 26, 2016

Owner

What's needed now is an API definition in order to use this feature. My day job has been keeping me busy and I haven't had the time to sit down and give it some thought. I should be freeing up mid-Jan, but right now time is pretty tight. Idea submissions are welcome!

Owner

damnhandy commented Dec 26, 2016

What's needed now is an API definition in order to use this feature. My day job has been keeping me busy and I haven't had the time to sit down and give it some thought. I should be freeing up mid-Jan, but right now time is pretty tight. Idea submissions are welcome!

@jroper

This comment has been minimized.

Show comment Hide comment
@jroper

jroper Jan 5, 2017

Hi @damnhandy, I'm just doing some exploratory work at the moment for Lagom Framework. Lagom provides a symmetric client/service interface, where a single interface is defined, and clients use it to speak to the server, and servers implement it to receive requests from the client. We have a single URI spec that both the server uses to extract parameters and the client uses to build URIs. We're looking at whether we should switch to RFC6570 templates, tracking the work here. So we'd definitely need this feature if we did. We'll take a look at the API and see if it's going to be suited to what we need - performance is one thing we're concerned about.

jroper commented Jan 5, 2017

Hi @damnhandy, I'm just doing some exploratory work at the moment for Lagom Framework. Lagom provides a symmetric client/service interface, where a single interface is defined, and clients use it to speak to the server, and servers implement it to receive requests from the client. We have a single URI spec that both the server uses to extract parameters and the client uses to build URIs. We're looking at whether we should switch to RFC6570 templates, tracking the work here. So we'd definitely need this feature if we did. We'll take a look at the API and see if it's going to be suited to what we need - performance is one thing we're concerned about.

@kag0

This comment has been minimized.

Show comment Hide comment
@kag0

kag0 Jan 6, 2017

@jroper I think it'd be great to have URI template adoption in Lagom, and input/contribution from lightbend to get this feature released would be awesome as well. For me I'd be down to write some more code if needed to get this complete.
I saw your comments on the PR, looks like good input.

kag0 commented Jan 6, 2017

@jroper I think it'd be great to have URI template adoption in Lagom, and input/contribution from lightbend to get this feature released would be awesome as well. For me I'd be down to write some more code if needed to get this complete.
I saw your comments on the PR, looks like good input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment