UriTemplate does not escape semicolons in path segments [SPR-11652]

[spring-projects-issues]

Benjamin Gehrels opened SPR-11652 and commented

In URIs (and URLs), path components may have parameters delimited by a semicolon (called path parameters or matrix parameters). Therefore semicolons contained in the variable values should be escaped when expanding variables in path components of UriTemplates. Otherwise, parts of the variables value will be interpreted as a path parameter (matrix variables) when parsing the URL later on.

I attached a unit test showing this problem:

new UriTemplate("http://www.example.com/user/{userId}/dashboard").expand("john;doe");

should return http://www.example.com/user/john%3Bdoe/dashboard but instead returns http://www.example.com/user/john;doe/dashboard. So, instead of john;doe's dashboard, john's dashboard is delivered with a parameter doe.

---

Affects: 4.0.2

Attachments:

• SpringUriTemplateSemicolonTest.java (523 bytes)

Issue Links:

• Support advanced URI Template Syntax [SPR-10505] #15137 Support advanced URI Template Syntax
• url param containing ? is not getting encoded by Rest Template [SPR-14101] #18673 url param containing ? is not getting encoded by Rest Template

Referenced from: commits 6f2c968

1 votes, 3 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

We don't escape ";" since it is a legal character in a path segment. You may for example be appending one or more path variables through a URI variable placeholder. Also as a general delimiter ";" may be used for example to provide multiple ids (e.g. "/foo/id1;id2;id3") so if expanding this as a URI variable ("/foo/{ids}") you would not want to encode the semicolon.

That said I realize this makes your case harder than it needs to be so I'm going to accept this as an issue. To resolve it we need to come up with a way to express your intent.

[spring-projects-issues]

Benjamin Gehrels commented

I think i get your point, that it's important to be able to set Path params using the Templating mechanism. The problem i see is, that - by allowing the templating mechanism to be used like this - it is impossible to set well escaped Semicolons in Path components.

If you write

new UriTemplate("http://www.example.com/user/{userId}/dashboard").expand("john;doe");

you get http://www.example.com/user/john;doe/dashboard, so doe will be interpreted as a path variable.

If your decide to pre-escape the semicolon to avoid this

new UriTemplate("http://www.example.com/user/{userId}/dashboard").expand("john%3Ddoe");

you get http://www.example.com/user/john%253Ddoe/dashboard, because the given escape sequences do not stay untouched, but get double-escaped instead.

This leads to a black hole in the templating system: There is no way to express "Take this arbitrary String and put it in the Path, so that it will be well escaped and treated as one entity".

The best workaround i found so far is post-processing the generated URI, which is only possible if you make certain assumptions about it, namely that there are no intended semicolons in the rest of the URI (and it is pretty ugly):

URI.create(new UriTemplate("http://www.example.com/user/{userId}/dashboard").expand("john;doe").toString().replace(";", "%3D"))

[spring-projects-issues]

Rossen Stoyanchev commented

You get more control with UriComponentsBuilder (in particular notice the use of build(encoded)):

UriComponentsBuilder.fromUriString("http://www.example.com/user/{userId}/dashboard").build(true).expand("john%3Ddoe").toUri();

That said I do realize that means you have to encode all URI variables yourself so it's hardly ideal.

There is no way to express "Take this arbitrary String and put it in the Path, so that it will be well escaped and treated as one entity".

Agreed fully. We need to imagine ways to express this. Suggestions welcome!

[spring-projects-issues]

Rossen Stoyanchev commented

There is no way to express "Take this arbitrary String and put it in the Path, so that it will be well escaped and treated as one entity".

Delving more into what "well escaped and treated as one entity", if you look at UriUtils.encodePathSegment and subsequently HierarchicalUriComponents.Type.PATH_SEGMENT, you'll see that ";" is one of a number of sub-delimiters. What isn't quite clear yet is exactly is what new UriUtils method and HierarchicalUriComponents.Type needs to be added to support this.

[spring-projects-issues]

Ben Kiefer commented

We got bit by this today with something like the following request.

"http://www.example.com/user/%3Bblah" GET

The path variable gets decoded to ';blah' in our receiving controller (above) and then we turn around and pass it along to another endpoint with rest template. Ex:

"http://www.other.com/user/;blah" GET

This ends up generating a request where the ;blah is treated as a request param instead of a path variable.

This results in a 405 from the lower level endpoint because it doesn't want request params.

[spring-projects-issues]

Ben Kiefer commented

Ended up fixing the problem with a request interceptor that rewrites the path of the URI if there is a semicolon in it. Wrote a quick interceptor for the rest template, and then decorated/proxied the HttpRequest that came into the intereceptor and fed the decorated request along the chain. I'd prefer to have a "global" solution that was part of spring though.

[spring-projects-issues]

Rossen Stoyanchev commented

Since 4.2 the RestTemplate can be configured with a UriTemplateHandler that can customize how all URI templates are expanded. The DefaultUriTemplateHandler now provides a property that can be used to ensure a stricter encoding of URI variable values such that any values outside the reserved character set are encoded:

DefaultUriTemplateHandler handler = new DefaultUriTemplateHandler();
handler.setStrictEncoding(true);

RestTemplate restTemplate = new RestTemplate();
restTemplate.setUriTemplateHandler(handler);

// ... 

Ben Kiefer and BGehrels it would be great if you could give 4.3.0.BUILD-SNAPSHOT a try and confirm how well this mode works for you. Thanks!