Call relative and canonical path support #7839

seglo · 2017-09-17T16:20:37Z

Pull Request Checklist

Have you read How to write the perfect pull request?
Have you read through the contributor guidelines?
Have you signed the Lightbend CLA?
Have you [squashed your commits]? (Optional, but makes merge messages nicer.)
Have you added copyright headers to new files?
Have you checked that both Scala and Java APIs are updated?
Have you updated the documentation for both Scala and Java sections?
- I updated Java and Scala docs. Should this feature be added to an existing documentation page, a new page, or are the code docs sufficient?
- EDIT: Docs added.
Have you added tests for any changed functionality?

Helpful things

Fixes

n/a

Purpose

Add a feature to Call to generate a relative route given a RequestHeader. Using relative routes makes it easier to develop standalone Play! apps that can be hosted behind proxy's or other HTTP gateways regardless of their prefix route. Several examples include:

Hosting Play! apps behind a webserver proxy like DC/OS's adminrouter. I ran into this issue when trying to get Kafka Manager to work behind the DC/OS admin router.
When dynamically rendering stylesheets, you need asset links to be relative because they may end up getting served from different URLs by a CDN.

Background Context

Due to several issues[1] with java.nio.file.Path.relativize(Path other) I decided to roll my own implementation to create a canonical and relative path. I ported Python's posixpath implementation with a few minor additions to support routes that end without a trailing /.

The implementation is in Java and self-contained within the play.mvc.Call type. Two new public methods are available:

String relative(Http.RequestHeader requestHeader)
String canonical()

The relative implementation is based on relpath (from posixpath.py) and commonprefix (from genericpath.py) in the Python 2.7 os.path package. To canonicalize the URL first I ported the normpath implementation (from posixpath.py) also in the os.path package.

I also included a suite of tests to support variations of paths to relativize and canonicalize.

References

Feature proposal on the play-framework-dev list: https://groups.google.com/forum/#!topic/play-framework-dev/SCXWrbk1k-s

[1] https://bugs.openjdk.java.net/browse/JDK-6925169, https://stackoverflow.com/a/37749932/895309

gmethvin

I see you mentioned you looked at java.nio.file.Path but it would be good to document what was missing that caused us to need a completely custom implementation here.

gmethvin · 2017-09-18T21:11:03Z

framework/src/play/src/main/scala/play/api/mvc/Call.scala

+   * // == "../url"
+   * }}}
+   */
+  def relative()(implicit request: RequestHeader): String = this.relative(request.path)


I don't think we need the () here.

gmethvin · 2017-09-18T21:11:05Z

framework/src/play/src/main/java/play/mvc/Call.java

+        return relative(requestHeader.path());
+    }
+
+    protected String relative(String startPath) {


Maybe this should be a public method public String relativeTo(String path).

Also this seems like a lot of the logic here could be extracted into a utility method for use elsewhere. Something like PathUtils.relativize.

When changing this to a public method, it could be good to also improve documentation (with examples) for this method.

marcospereira · 2017-09-18T23:33:57Z

framework/src/play/src/main/java/play/mvc/Call.java

@@ -133,6 +139,111 @@ public String webSocketURL(boolean secure, String host) {
      return "ws" + (secure ? "s" : "") + "://" + host + this.url();
    }

+    private static String CURDIR = ".";


Can we have the full names here? CURRENT_DIR, SEPARATOR, etc.

jroper · 2017-09-18T23:40:19Z

framework/src/play/src/main/java/play/mvc/Call.java

+        String trailingSep = url.endsWith(SEP) ? SEP : "";
+
+        return prefixSep + canonical.stream().collect(Collectors.joining(SEP)) + trailingSep + this.appendFragment();
+    }


If this implementation is robust (I haven't looked at it carefully enough to know), then we should extract it out somewhere and use it to solve #2337.

seglo · 2017-09-20T01:04:53Z

Thanks for the review. I'll follow up with another commit shortly.

@gmethvin There are several reasons I chose to port Python's impl. instead of using the Java Class Library.

The directory separator depends on your platform. If run on Windows you would get a \\ separator. I don't believe there's any way to override this, so it would need to be replaced after.
java.nio.file.Path.relativize expects path's to be directories. This isn't a big deal, but it would require the same implementation I make now where I have to strip file's from both paths and re-add them later if applicable.
java.nio.file.Path.relativize cannot support .. or .. The path would need to be canonicalized. This could be done with java.io.File.getCanonicalFile or java.io.File.getCanonicalPath, but then you have to somehow handle checked exceptions for IOException.

So while it's possible to use the JCL at the end of the day (with some massaging) I just feel that it's not the appropriate thing to do given that the options available are meant to be used with the filesystem and not URL's. However, if you or the rest of the team feel that you would prefer that I can update the PR accordingly.

… relativeTo. Full names for dir constants. Dropped first params list from scala Call relative method.

gmethvin · 2017-09-21T03:44:15Z

framework/src/play/src/main/java/play/libs/Paths.java

+/*
+ * Copyright (C) 2009-2017 Lightbend Inc. <https://www.lightbend.com>
+ */
+package play.libs;


Let's put this in play.core.utils, since this is a utility mainly for internal use.

seglo · 2017-09-22T18:07:19Z

@gmethvin I updated it to the play.core.j namespace. Is that OK?

@marcospereira What documentation page and section would recommend updating? The main reference to Call is in the Routing pages (i.e. https://www.playframework.com/documentation/2.6.x/JavaRouting#Reverse-routing). There's also some refs in the JavaScript routing page (https://www.playframework.com/documentation/2.6.x/ScalaJavascriptRouting#router-resource). Although this feature isn't just for Js routing, maybe it could be mentioned there. WDYT?

gmethvin

@seglo play.core.j is a weird namespace to use, since that's typically used for Java adapters to the scala API. This looks like more of a generic utility to me.

gmethvin · 2017-09-22T20:39:02Z

framework/src/play/src/main/java/play/core/j/Paths.java

+/**
+ * Utilities to work with URL paths.
+ */
+public class Paths {


Maybe add a private default constructor here since this isn't meant to be instantiated.

seglo · 2017-09-22T20:59:36Z

@gmethvin Should I just create the play.core package under the java source tree?

gmethvin · 2017-09-22T21:01:59Z

@seglo Yes, you can do that, or if you prefer you can rewrite your code in scala (I don't have a strong opinion either way).

seglo · 2017-09-22T21:05:10Z

@gmethvin OK. I should have just written it in Scala to begin with.. I had tunnel vision because Call was defined in the java source tree. Will ponder over the weekend..

seglo · 2017-09-25T19:21:40Z

I kept it as Java because I didn't have a compelling reason to rewrite it. I refactored the test coverage as suggested and made other misc. updates.

seglo · 2017-09-28T15:00:50Z

LMK where you think the most appropriate place to document this feature would be and I'll add a commit.

The main reference to Call is in the Routing pages (i.e. https://www.playframework.com/documentation/2.6.x/JavaRouting#Reverse-routing). There's also some refs in the JavaScript routing page (https://www.playframework.com/documentation/2.6.x/ScalaJavascriptRouting#router-resource). Although this feature isn't just for Js routing, maybe it could be mentioned there. WDYT?

gmethvin · 2017-10-01T07:01:27Z

@seglo I think the reverse routing section is the proper place to document it. Perhaps it would be good to add an example with a template showing both absolute and relative variants.

seglo · 2017-10-05T02:06:08Z

I've added documentation to the Java and Scala guide's Routing sections. This was the last outstanding commit I had.

seglo · 2017-10-06T01:07:42Z

@gmethvin I fixed some of the errors from the documentation test build, but I'm not really sure what's going on the remaining errors. Could you assist?

gmethvin · 2017-10-09T02:24:46Z

@seglo There were a couple problems. First is that the main template doesn't exist (maybe you forgot to commit it?). The directories also didn't match the package names, and the way twirl templates work it will automatically infer the package name from the directory.

See gmethvin@24b6713

seglo · 2017-10-09T20:41:27Z

Thanks a ton @gmethvin . I was getting mixed up between the doc package namespaces and the code sample namespaces. I doublechecked the rendered output and everything looks good. Your commit fixed the failing documentation tests.

I think we're ready for a final review. Thanks for your patience.

gmethvin

lgtm

* Call relative and canonical path support * Move relative & canonical impl to Paths util type. Rename relative -> relativeTo. Full names for dir constants. Dropped first params list from scala Call relative method. * Move Paths to play.core.j namespace * Moved to play.core package * Re-factor tests * Relative routes documentation * Docs tests fixes * Fix package structure * Fix refs to code samples

seglo force-pushed the relative-routes branch from 05d7d4b to 7a6d134 Compare September 18, 2017 00:58

Call relative and canonical path support

9aee6c7

seglo force-pushed the relative-routes branch from 7a6d134 to 9aee6c7 Compare September 18, 2017 01:00

gmethvin reviewed Sep 18, 2017

View reviewed changes

marcospereira reviewed Sep 18, 2017

View reviewed changes

jroper reviewed Sep 18, 2017

View reviewed changes

Move relative & canonical impl to Paths util type. Rename relative ->…

df932c5

… relativeTo. Full names for dir constants. Dropped first params list from scala Call relative method.

gmethvin reviewed Sep 21, 2017

View reviewed changes

Move Paths to play.core.j namespace

967b7a7

gmethvin reviewed Sep 22, 2017

View reviewed changes

seglo added 2 commits September 25, 2017 14:37

Moved to play.core package

cf5dbc3

Re-factor tests

780e6cf

Relative routes documentation

f4625c0

Docs tests fixes

1bd6b94

gmethvin and others added 2 commits October 9, 2017 16:22

Fix package structure

dd02527

Fix refs to code samples

d5321c0

gmethvin approved these changes Oct 20, 2017

View reviewed changes

gmethvin merged commit e74fe3d into playframework:master Oct 20, 2017

marcospereira added type:improvement type:feature labels Oct 23, 2017

marcospereira added this to the Play 2.6.7 milestone Oct 23, 2017

Call relative and canonical path support #7839

Call relative and canonical path support #7839

seglo commented Sep 17, 2017 •

edited

gmethvin left a comment •

edited

gmethvin Sep 18, 2017

gmethvin Sep 18, 2017

gmethvin Sep 18, 2017

marcospereira Sep 18, 2017

marcospereira Sep 18, 2017

jroper Sep 18, 2017 •

edited

seglo commented Sep 20, 2017 •

edited

gmethvin Sep 21, 2017

seglo commented Sep 22, 2017

gmethvin left a comment

gmethvin Sep 22, 2017

seglo commented Sep 22, 2017

gmethvin commented Sep 22, 2017

seglo commented Sep 22, 2017

seglo commented Sep 25, 2017

seglo commented Sep 28, 2017

gmethvin commented Oct 1, 2017

seglo commented Oct 5, 2017

seglo commented Oct 6, 2017

gmethvin commented Oct 9, 2017 •

edited

seglo commented Oct 9, 2017 •

edited

gmethvin left a comment

Call relative and canonical path support #7839

Call relative and canonical path support #7839

Conversation

seglo commented Sep 17, 2017 • edited

Pull Request Checklist

Helpful things

Fixes

Purpose

Background Context

References

gmethvin left a comment • edited

Choose a reason for hiding this comment

gmethvin Sep 18, 2017

Choose a reason for hiding this comment

gmethvin Sep 18, 2017

Choose a reason for hiding this comment

gmethvin Sep 18, 2017

Choose a reason for hiding this comment

marcospereira Sep 18, 2017

Choose a reason for hiding this comment

marcospereira Sep 18, 2017

Choose a reason for hiding this comment

jroper Sep 18, 2017 • edited

Choose a reason for hiding this comment

seglo commented Sep 20, 2017 • edited

gmethvin Sep 21, 2017

Choose a reason for hiding this comment

seglo commented Sep 22, 2017

gmethvin left a comment

Choose a reason for hiding this comment

gmethvin Sep 22, 2017

Choose a reason for hiding this comment

seglo commented Sep 22, 2017

gmethvin commented Sep 22, 2017

seglo commented Sep 22, 2017

seglo commented Sep 25, 2017

seglo commented Sep 28, 2017

gmethvin commented Oct 1, 2017

seglo commented Oct 5, 2017

seglo commented Oct 6, 2017

gmethvin commented Oct 9, 2017 • edited

seglo commented Oct 9, 2017 • edited

gmethvin left a comment

Choose a reason for hiding this comment

seglo commented Sep 17, 2017 •

edited

gmethvin left a comment •

edited

jroper Sep 18, 2017 •

edited

seglo commented Sep 20, 2017 •

edited

gmethvin commented Oct 9, 2017 •

edited

seglo commented Oct 9, 2017 •

edited