Use HttpApp instead of HttpRoutes in Http4sServlet #3012

CharlesD91 · 2019-12-06T20:07:03Z

Same NotFound behavior as with HttpRoutes for unspecified endpoints (see NotFoundHttpAppSpec)
Preserve retrocompatibility in JettyBuilder, ServletContextSyntax, TomcatBuilder

rossabaker

Looks good to me, with that name tweak.

Pinging @nigredo-tori, who has worked a lot on the servlets and may have commentary on the breaking change, but I think it's a good idea.

rossabaker · 2019-12-11T01:07:05Z

tomcat/src/main/scala/org/http4s/server/tomcat/TomcatBuilder.scala

@@ -120,6 +121,9 @@ sealed class TomcatBuilder[F[_]] private (
    })

  def mountService(service: HttpRoutes[F], prefix: String): Self =
+    mountServiceHttpApp(http4s.httpRoutesToApp(service), prefix)
+
+  def mountServiceHttpApp(service: HttpApp[F], prefix: String): Self =


I think I would just call this mountHttpApp.

On blaze, it's called witHttpApp, but the servlet model is a little different: we have more than one. So I think the different name is justified.

nigredo-tori

breaking change

The only breaking change I see here is that the parameters for the servlet constructors and apply methods have changed. Ideally, we should provide additional @deprecated constructors and methods with old signatures to avoid breaking packages, compiled against previous 0.21 milestones - although I don't think many libraries use those directly.

nigredo-tori · 2019-12-11T01:51:26Z

servlet/src/main/scala/org/http4s/servlet/AsyncHttp4sServlet.scala

          .suspend(serviceFn(request))
-          .getOrElse(Response.notFound)
+          .recover({ case _: scala.MatchError => Response.notFound[F] })


We probably don't need this. serviceFn is a total function, so any MatchErrors are implementation errors, and shouldn't be handled (and especially swallowed) here. This also means we don't need to test this.

Same in BlockingHttp4sServlet.

It would be nice to be able to handle NotFound in a single place, that's why I used a PartialFunction and then handle the NotFound in the recover. Otherwise, when you have multiple services, you have to add case _ => handleNotFound() at the end of each of them.

Furthermore, if you don't mount a service on the prefix /, then a request on anything caught by / will return an HTML that doesn't seem to be programmable (even though fixing this would require to modify the code in JettyBuilder.resource() which I haven't done here but could be factorized in a single place with what I describe in the above paragraph).

Sorry, I'm not quite sure what you're getting at here. I'll try and answer some of the points you are trying to make...

First, it looks like you've added this recover as a way to solve some issue in JettyBuilder, and it handles the situation that should not happen during any normal use of the AsyncHttp4sServer. This smells.

Second, in the Cats world we make a lot of assumptions, one of which is that all the functions are total except for PartialFunctions. We can only get a MatchError by breaking this rule via a non-exhaustive match - which, by the way, would lead to a compiler warning. Generally, any MatchError thrown signals broken code, and we should never try specifically handling it for an arbitrary function provided by the user code, never mind swallowing it without reporting. Basically, if you want to support a non-exhaustive service - use HttpRoutes instead. This is exactly what it is there for.

Third, regarding case _ => handleNotFound() - we can do .orNotFound when mounting HttpRoutes in JettyBuilder. I'm not sure why that would be a problem.

Finally, regarding /. As I understand the problem (if you can call it that) is that Jetty catches any requests that were not handled, and shows a standard 404 error page. I'm not sure it's our place to change that by default, but even if it were - as I understand it, this can be solved by mounting a servlet to /. I'm still not sure what all of this has to do with the servlet implementation, though.

Ok thank you for your comments. I think I understand your points. I will try to update my PR accordingly.

@nigredo-tori I added def orNotFound(notFoundHandler: A => Response[F]): Kleisli[F, A, Response[F]] to KleisliSyntax. Also I updated NotFoundHttpAppSpec. It describes what I am trying to achieve here: having a simple, single way of handling custom NotFound responses accross the server. Do you think this would be reasonable ?

(also I agree with your latest comment about simply adding

def mountHttpApp(service: HttpApp[F], prefix: String): Self = mountService(service.mapK(OptionT.liftK[F])), prefix)

to the builders)

I've tried and summarized the important points of this below, and I suggest we continue this discussion there, since the scope of it went far beyond the single line of code I want gone. 😄

nigredo-tori · 2019-12-11T01:54:58Z

tomcat/src/main/scala/org/http4s/server/tomcat/TomcatBuilder.scala

@@ -120,6 +121,9 @@ sealed class TomcatBuilder[F[_]] private (
    })

  def mountService(service: HttpRoutes[F], prefix: String): Self =
+    mountServiceHttpApp(http4s.httpRoutesToApp(service), prefix)


We're in package org.http4s, so we can just use httpRoutesToApp(service).

nigredo-tori · 2019-12-11T02:15:05Z

jetty/src/test/scala/org/http4s/server/jetty/Issue454.scala

@@ -44,7 +44,7 @@ object Issue454 {
  }

  val servlet = new AsyncHttp4sServlet[IO](
-    service = HttpRoutes.of[IO] {
+    service = HttpApp {
      case GET -> Root => Ok(insanelyHugeData)
    },


Please try and avoid non-exhaustive matches. This can be done with httpRoutesToApp, or by adding a catch-all clause.

use the .orNotFound syntax in as described below

nigredo-tori · 2019-12-11T03:10:36Z

core/src/main/scala/org/http4s/package.scala

@@ -77,4 +78,7 @@ package object http4s { // scalastyle:ignore
  type Http4sSyntax = syntax.AllSyntax
  @deprecated("Moved to org.http4s.syntax.all", "0.16")
  val Http4sSyntax = syntax.all
+
+  def httpRoutesToApp[F[_]: ConcurrentEffect](httpRoutes: HttpRoutes[F]): HttpApp[F] =
+    HttpApp[F](req => httpRoutes(req).getOrElse[Response[F]](Response.notFound[F]))


We already have a syntax for this. With import org.http4s.syntax.kleisli._ we can do httpRoutes.orNotFound. So I don't think we need this as a separate function.

nigredo-tori · 2019-12-13T09:29:14Z

Actually, if the motivation behind this PR is to just add mountHttpApp to the builders - can't we just add something like this to JettyBuilder and TomcatBuilder?

def mountHttpApp(service: HttpApp[F], prefix: String): Self =
    mountService(service.mapK(OptionT.liftK[F])), prefix)

This way we wouldn't have to break anything.

Unless, of course, the change to the servlets is the main goal - in which case I don't see the point of it, since we can always get a HttpApp from HttpRoutes with .orNotFound.

nigredo-tori · 2019-12-13T14:06:13Z

So, to summarize the discussion (and to get my thoughts in order):

You want to add a way to mount HttpApps to Jetty and Tomcat builders via mountHttpApp. I think this is useful, and it should be done.
The easiest way to do this is to add mountHttpApp that factors through mountService, as I proposed before. However, in this case the HttpApp would be transformed to HttpRoutes, just to be later made back into a HttpApp in the servlet implementation via .orNotFound. This is a bit silly.
Looking at the servlet implementations, it makes sense for them to receive a HttpApp instead of HttpRoutes they get now, since the Servlet interface itself implies that every request should get a response. It makes sense to move .orNotFound calls outside the servlet implementations, since this adds flexibility. So you've changed the constructors for servlet implementations to use HttpApp (so that mountService factors through mountHttpApp). Even though this is a breaking change, it shouldn't be hard to fix in the calling code (a single .orNotFound), and there's probably not a lot of such code out there. All of this seems completely reasonable to me.
The recover { case _: MatchError => thing is a hack to make the tests compile without properly updating them. It should be removed.
The httpRoutesToApp function is unnecessary since its use case is already covered by the .orNotFound syntax.
Regarding the tests for all of the above, I don't think we need any beyond updating the ones that were already there. This includes fixing any non-exhaustive matches.

nigredo-tori · 2019-12-13T14:22:03Z

core/src/main/scala/org/http4s/syntax/KleisliSyntax.scala

@@ -30,6 +30,9 @@ trait KleisliSyntaxBinCompat1 {
 final class KleisliResponseOps[F[_]: Functor, A](self: Kleisli[OptionT[F, ?], A, Response[F]]) {
  def orNotFound: Kleisli[F, A, Response[F]] =
    Kleisli(a => self.run(a).getOrElse(Response.notFound))
+
+  def orNotFound(notFoundHandler: A => Response[F]): Kleisli[F, A, Response[F]] =
+    Kleisli(a => self.run(a).getOrElse(notFoundHandler(a)))


I don't think this use case (a custom fall-through handler) is common enough to warrant special syntax. I can also think of a few other ways to implement similar functionality - instead of A => Response[F] we can use Response[F], F[Response[F]], A => F[Response[F]], HttpApp[F]... All of those potential functions are equally valid, and would have their uses - I don't think the one you've chosen is the only one that should have a syntax.

If you insist, I'd like you to at least choose another name. Overloading should be avoided in general, and orNotFound is misleading here, since it has no relation to NotFound.

It would be nice to be able to do:

JettyBuilder[IO] .bindAny() .mountServiceHttpApp( httpRoutes0.orNotFound(myCustomNotFoundHandler), "/" ) .mountServiceHttpApp( httpRoutes1.orNotFound(myCustomNotFoundHandler), "/prefix1" ) (...) .mountServiceHttpApp( httpRoutesN.orNotFound(myCustomNotFoundHandler), "/prefixN" )

So that you can see in one place how all non-matched endpoints are handled and that there are no leaks. The method name could be .orHandleNonMatch. I agree with all your other points.

So that you can see in one place how all non-matched endpoints are handled and that there are no leaks.

You don't need a (library) syntax for that - it works just as well with a custom function:

def orMyHandler(x: HttpRoutes[F]): HttpApp[F] = Kleisli { req => x.run(req).getOrElse(myCustomNotFoundHandler(req)) } //... .mountServiceHttpApp( orMyHandler(httpRoutes0), "/" ) // ...

As an aside, I've never seen someone mount a lot of services onto one builder like this. I'd rewrite your example like this instead:

JettyBuilder[IO] .bindAny() .mountServiceHttpApp( myOrHandler( Router( "" -> httpRoutes0, "prefix1" -> httpRoutes1, ... "prefixN" -> httpRoutesN ) ), "/" )

Thank you, this is exactly what I have been looking for !

More generally, do you think there is a need for a JettyBuilder.mountServiceHttpApp ? I would say yes because as .mountServiceHttpApp expects a total function then it is clearer to see how NotFound for instance are handled. Also it would make sense to unify the APIs between Jetty and Blaze. What do you think ?

Added clarity is not as important here; AFAICT, people tend to separate their routing logic from the server itself (http4s is great in that), so whoever wants to be sure they have matched all the possible options is probably already using HttpApp anyway, and just rewraps that as HttpRoutes. I think mountServiceHttpApp (or, rather, mountHttpApp) will be useful chiefly because it would allow us to avoid that annoying rewrapping.

Also it would make sense to unify the APIs between Jetty and Blaze.

Not really. As it was mentioned here, Jetty and Blaze builders have fundamentally different capabilities for mounting routes, so I don't see an attempt to unify them leading to anything other than extra complexity and confusion.

I think mountServiceHttpApp (or, rather, mountHttpApp) will be useful chiefly because it would allow us to avoid that annoying rewrapping.

Avoiding useless rewrapping on the library-user side is always good, I will simplify this PR and follow your advices

nigredo-tori

👍 once these two minor things are addressed.

nigredo-tori · 2019-12-18T10:35:46Z

servlet/src/test/scala/org/http4s/servlet/BlockingHttp4sServletSpec.scala

@@ -14,7 +14,7 @@ import scala.concurrent.duration._

 class BlockingHttp4sServletSpec extends Http4sSpec {

-  lazy val service = HttpRoutes.of[IO] {
+  lazy val service = HttpApp[IO] {


This match is now non-exhaustive! Please add .orNotFound below.

This was fixed in my latest commit

nigredo-tori · 2019-12-18T10:40:12Z

jetty/src/test/scala/org/http4s/server/jetty/NotFoundHttpAppSpec.scala

+import scala.io.Source
+import scala.util.Try
+
+class NotFoundHttpAppSpec extends Http4sSpec {


We no longer need this spec - it doesn't test anything that isn't covered elsewhere.

Ok, I removed it

CharlesD91 · 2019-12-18T10:44:43Z

Hello @nigredo-tori , I fixed the PR following your advice.

For some reason Travis CI keeps failing on 1 of the 3 test machines:

[error] ## Exception when compiling 2 sources to /home/travis/build/http4s/http4s/json4s-jackson/target/scala-2.13/classes
[error] scala.reflect.internal.FatalError: Error accessing /home/travis/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.9.8/jackson-databind-2.9.8.jar

but it is successful on the 2 others.

Please let me know what you think and thank you for your help !

nigredo-tori · 2019-12-18T11:13:04Z

I kept NotFoundHttpAppSpec because I think it is good to have a test on JettyBuilder.mountHttpApp.

We already test it in JettyServerSpec since mountService factors through it.

Also it displays how NotFound can be handled, while it is handled by default when using JettyBuilder.mountService.

This by itself is not a sufficient reason for adding tests. Also, the most common case for making a HttpApp from HttpRoutes (.orNotFound) is already used extensively throughout the documentation. Manually handling this depends on the use case, and should be reasonably easy to arrive to for people familiar with Kleisli and OptionT, just by looking at HttpRoutes and HttpApp types.

nigredo-tori · 2019-12-18T11:18:57Z

👍 from me. @hamnis, @rossabaker, please take a look again - this PR has gotten a lot lighter.

hamnis · 2019-12-19T11:33:46Z

I cleared the travis cache and restarted the build, can be merged on build success.

CharlesD91 force-pushed the http_app_jetty branch 4 times, most recently from 36d7d65 to fcebca0 Compare December 6, 2019 21:46

Use HttpApp in Http4sServlet

fb43c0d

CharlesD91 force-pushed the http_app_jetty branch from fcebca0 to fb43c0d Compare December 10, 2019 12:20

rossabaker reviewed Dec 11, 2019

View reviewed changes

nigredo-tori suggested changes Dec 11, 2019

View reviewed changes

nigredo-tori reviewed Dec 11, 2019

View reviewed changes

Use HttpApp in Http4sServlet

82913d6

nigredo-tori reviewed Dec 13, 2019

View reviewed changes

CharlesD91 added 2 commits December 18, 2019 11:19

Use HttpApp in Http4sServlet

bc0b8c4

Use HttpApp in Http4sServlet

c3de63e

nigredo-tori approved these changes Dec 18, 2019

View reviewed changes

Use HttpApp in Http4sServlet

f120a17

hamnis approved these changes Dec 19, 2019

View reviewed changes

hamnis merged commit f1204a2 into http4s:master Dec 20, 2019

CharlesD91 mentioned this pull request Dec 20, 2019

Parametrize NotFound #3002

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use HttpApp instead of HttpRoutes in Http4sServlet #3012

Use HttpApp instead of HttpRoutes in Http4sServlet #3012

CharlesD91 commented Dec 6, 2019

rossabaker left a comment

rossabaker Dec 11, 2019

nigredo-tori left a comment

nigredo-tori Dec 11, 2019 •

edited

CharlesD91 Dec 12, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

CharlesD91 Dec 13, 2019

CharlesD91 Dec 13, 2019

nigredo-tori Dec 13, 2019

nigredo-tori Dec 11, 2019

nigredo-tori Dec 11, 2019 •

edited

hamnis Dec 12, 2019

nigredo-tori Dec 11, 2019 •

edited

nigredo-tori commented Dec 13, 2019 •

edited

nigredo-tori commented Dec 13, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

CharlesD91 Dec 13, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

CharlesD91 Dec 13, 2019

nigredo-tori Dec 13, 2019

CharlesD91 Dec 13, 2019 •

edited

nigredo-tori left a comment •

edited

nigredo-tori Dec 18, 2019

CharlesD91 Dec 18, 2019

nigredo-tori Dec 18, 2019

CharlesD91 Dec 18, 2019

CharlesD91 commented Dec 18, 2019 •

edited

nigredo-tori commented Dec 18, 2019

nigredo-tori commented Dec 18, 2019

hamnis commented Dec 19, 2019

Use HttpApp instead of HttpRoutes in Http4sServlet #3012

Use HttpApp instead of HttpRoutes in Http4sServlet #3012

Conversation

CharlesD91 commented Dec 6, 2019

rossabaker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nigredo-tori left a comment

Choose a reason for hiding this comment

nigredo-tori Dec 11, 2019 • edited

Choose a reason for hiding this comment

CharlesD91 Dec 12, 2019 • edited

Choose a reason for hiding this comment

nigredo-tori Dec 13, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nigredo-tori Dec 11, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nigredo-tori Dec 11, 2019 • edited

Choose a reason for hiding this comment

nigredo-tori commented Dec 13, 2019 • edited

nigredo-tori commented Dec 13, 2019 • edited

nigredo-tori Dec 13, 2019 • edited

Choose a reason for hiding this comment

CharlesD91 Dec 13, 2019 • edited

Choose a reason for hiding this comment

nigredo-tori Dec 13, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CharlesD91 Dec 13, 2019 • edited

Choose a reason for hiding this comment

nigredo-tori left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CharlesD91 commented Dec 18, 2019 • edited

nigredo-tori commented Dec 18, 2019

nigredo-tori commented Dec 18, 2019

hamnis commented Dec 19, 2019

nigredo-tori Dec 11, 2019 •

edited

CharlesD91 Dec 12, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

nigredo-tori Dec 11, 2019 •

edited

nigredo-tori Dec 11, 2019 •

edited

nigredo-tori commented Dec 13, 2019 •

edited

nigredo-tori commented Dec 13, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

CharlesD91 Dec 13, 2019 •

edited

nigredo-tori Dec 13, 2019 •

edited

CharlesD91 Dec 13, 2019 •

edited

nigredo-tori left a comment •

edited

CharlesD91 commented Dec 18, 2019 •

edited