Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use HttpApp instead of HttpRoutes in Http4sServlet #3012

Merged
merged 5 commits into from Dec 20, 2019

Conversation

CharlesD91
Copy link
Contributor

@CharlesD91 CharlesD91 commented Dec 6, 2019

  • Same NotFound behavior as with HttpRoutes for unspecified endpoints (see NotFoundHttpAppSpec)

  • Preserve retrocompatibility in JettyBuilder, ServletContextSyntax, TomcatBuilder

@CharlesD91 CharlesD91 force-pushed the http_app_jetty branch 4 times, most recently from 36d7d65 to fcebca0 Compare Dec 6, 2019
Copy link
Member

@rossabaker rossabaker left a comment

Looks good to me, with that name tweak.

Pinging @nigredo-tori, who has worked a lot on the servlets and may have commentary on the breaking change, but I think it's a good idea.

@@ -120,6 +121,9 @@ sealed class TomcatBuilder[F[_]] private (
})

def mountService(service: HttpRoutes[F], prefix: String): Self =
mountServiceHttpApp(http4s.httpRoutesToApp(service), prefix)

def mountServiceHttpApp(service: HttpApp[F], prefix: String): Self =
Copy link
Member

@rossabaker rossabaker Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would just call this mountHttpApp.

On blaze, it's called witHttpApp, but the servlet model is a little different: we have more than one. So I think the different name is justified.

Copy link
Contributor

@nigredo-tori nigredo-tori left a comment

breaking change

The only breaking change I see here is that the parameters for the servlet constructors and apply methods have changed. Ideally, we should provide additional @deprecated constructors and methods with old signatures to avoid breaking packages, compiled against previous 0.21 milestones - although I don't think many libraries use those directly.

.suspend(serviceFn(request))
.getOrElse(Response.notFound)
.recover({ case _: scala.MatchError => Response.notFound[F] })
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need this. serviceFn is a total function, so any MatchErrors are implementation errors, and shouldn't be handled (and especially swallowed) here. This also means we don't need to test this.

Same in BlockingHttp4sServlet.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to be able to handle NotFound in a single place, that's why I used a PartialFunction and then handle the NotFound in the recover. Otherwise, when you have multiple services, you have to add case _ => handleNotFound() at the end of each of them.

Furthermore, if you don't mount a service on the prefix /, then a request on anything caught by / will return an HTML that doesn't seem to be programmable (even though fixing this would require to modify the code in JettyBuilder.resource() which I haven't done here but could be factorized in a single place with what I describe in the above paragraph).

Copy link
Contributor

@nigredo-tori nigredo-tori Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm not quite sure what you're getting at here. I'll try and answer some of the points you are trying to make...

First, it looks like you've added this recover as a way to solve some issue in JettyBuilder, and it handles the situation that should not happen during any normal use of the AsyncHttp4sServer. This smells.

Second, in the Cats world we make a lot of assumptions, one of which is that all the functions are total except for PartialFunctions. We can only get a MatchError by breaking this rule via a non-exhaustive match - which, by the way, would lead to a compiler warning. Generally, any MatchError thrown signals broken code, and we should never try specifically handling it for an arbitrary function provided by the user code, never mind swallowing it without reporting. Basically, if you want to support a non-exhaustive service - use HttpRoutes instead. This is exactly what it is there for.

Third, regarding case _ => handleNotFound() - we can do .orNotFound when mounting HttpRoutes in JettyBuilder. I'm not sure why that would be a problem.

Finally, regarding /. As I understand the problem (if you can call it that) is that Jetty catches any requests that were not handled, and shows a standard 404 error page. I'm not sure it's our place to change that by default, but even if it were - as I understand it, this can be solved by mounting a servlet to /. I'm still not sure what all of this has to do with the servlet implementation, though.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thank you for your comments. I think I understand your points. I will try to update my PR accordingly.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nigredo-tori I added def orNotFound(notFoundHandler: A => Response[F]): Kleisli[F, A, Response[F]] to KleisliSyntax. Also I updated NotFoundHttpAppSpec. It describes what I am trying to achieve here: having a simple, single way of handling custom NotFound responses accross the server. Do you think this would be reasonable ?

(also I agree with your latest comment about simply adding

def mountHttpApp(service: HttpApp[F], prefix: String): Self =
    mountService(service.mapK(OptionT.liftK[F])), prefix)

to the builders)

Copy link
Contributor

@nigredo-tori nigredo-tori Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried and summarized the important points of this below, and I suggest we continue this discussion there, since the scope of it went far beyond the single line of code I want gone. 😄

@@ -120,6 +121,9 @@ sealed class TomcatBuilder[F[_]] private (
})

def mountService(service: HttpRoutes[F], prefix: String): Self =
mountServiceHttpApp(http4s.httpRoutesToApp(service), prefix)
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're in package org.http4s, so we can just use httpRoutesToApp(service).

@@ -44,7 +44,7 @@ object Issue454 {
}

val servlet = new AsyncHttp4sServlet[IO](
service = HttpRoutes.of[IO] {
service = HttpApp {
case GET -> Root => Ok(insanelyHugeData)
},
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please try and avoid non-exhaustive matches. This can be done with httpRoutesToApp, or by adding a catch-all clause.

Copy link
Contributor

@hamnis hamnis Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use the .orNotFound syntax in as described below

@@ -77,4 +78,7 @@ package object http4s { // scalastyle:ignore
type Http4sSyntax = syntax.AllSyntax
@deprecated("Moved to org.http4s.syntax.all", "0.16")
val Http4sSyntax = syntax.all

def httpRoutesToApp[F[_]: ConcurrentEffect](httpRoutes: HttpRoutes[F]): HttpApp[F] =
HttpApp[F](req => httpRoutes(req).getOrElse[Response[F]](Response.notFound[F]))
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a syntax for this. With import org.http4s.syntax.kleisli._ we can do httpRoutes.orNotFound. So I don't think we need this as a separate function.

@nigredo-tori
Copy link
Contributor

@nigredo-tori nigredo-tori commented Dec 13, 2019

Actually, if the motivation behind this PR is to just add mountHttpApp to the builders - can't we just add something like this to JettyBuilder and TomcatBuilder?

def mountHttpApp(service: HttpApp[F], prefix: String): Self =
    mountService(service.mapK(OptionT.liftK[F])), prefix)

This way we wouldn't have to break anything.

Unless, of course, the change to the servlets is the main goal - in which case I don't see the point of it, since we can always get a HttpApp from HttpRoutes with .orNotFound.

@nigredo-tori
Copy link
Contributor

@nigredo-tori nigredo-tori commented Dec 13, 2019

So, to summarize the discussion (and to get my thoughts in order):

  1. You want to add a way to mount HttpApps to Jetty and Tomcat builders via mountHttpApp. I think this is useful, and it should be done.
  2. The easiest way to do this is to add mountHttpApp that factors through mountService, as I proposed before. However, in this case the HttpApp would be transformed to HttpRoutes, just to be later made back into a HttpApp in the servlet implementation via .orNotFound. This is a bit silly.
  3. Looking at the servlet implementations, it makes sense for them to receive a HttpApp instead of HttpRoutes they get now, since the Servlet interface itself implies that every request should get a response. It makes sense to move .orNotFound calls outside the servlet implementations, since this adds flexibility. So you've changed the constructors for servlet implementations to use HttpApp (so that mountService factors through mountHttpApp). Even though this is a breaking change, it shouldn't be hard to fix in the calling code (a single .orNotFound), and there's probably not a lot of such code out there. All of this seems completely reasonable to me.
  4. The recover { case _: MatchError => thing is a hack to make the tests compile without properly updating them. It should be removed.
  5. The httpRoutesToApp function is unnecessary since its use case is already covered by the .orNotFound syntax.
  6. Regarding the tests for all of the above, I don't think we need any beyond updating the ones that were already there. This includes fixing any non-exhaustive matches.

@@ -30,6 +30,9 @@ trait KleisliSyntaxBinCompat1 {
final class KleisliResponseOps[F[_]: Functor, A](self: Kleisli[OptionT[F, ?], A, Response[F]]) {
def orNotFound: Kleisli[F, A, Response[F]] =
Kleisli(a => self.run(a).getOrElse(Response.notFound))

def orNotFound(notFoundHandler: A => Response[F]): Kleisli[F, A, Response[F]] =
Kleisli(a => self.run(a).getOrElse(notFoundHandler(a)))
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this use case (a custom fall-through handler) is common enough to warrant special syntax. I can also think of a few other ways to implement similar functionality - instead of A => Response[F] we can use Response[F], F[Response[F]], A => F[Response[F]], HttpApp[F]... All of those potential functions are equally valid, and would have their uses - I don't think the one you've chosen is the only one that should have a syntax.

If you insist, I'd like you to at least choose another name. Overloading should be avoided in general, and orNotFound is misleading here, since it has no relation to NotFound.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to be able to do:

JettyBuilder[IO]
      .bindAny()
      .mountServiceHttpApp(
        httpRoutes0.orNotFound(myCustomNotFoundHandler),
        "/"
      )
      .mountServiceHttpApp(
        httpRoutes1.orNotFound(myCustomNotFoundHandler),
        "/prefix1"
      )
(...)
      .mountServiceHttpApp(
        httpRoutesN.orNotFound(myCustomNotFoundHandler),
        "/prefixN"
      )

So that you can see in one place how all non-matched endpoints are handled and that there are no leaks. The method name could be .orHandleNonMatch. I agree with all your other points.

Copy link
Contributor

@nigredo-tori nigredo-tori Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that you can see in one place how all non-matched endpoints are handled and that there are no leaks.

You don't need a (library) syntax for that - it works just as well with a custom function:

def orMyHandler(x: HttpRoutes[F]): HttpApp[F] =
  Kleisli { req => x.run(req).getOrElse(myCustomNotFoundHandler(req)) }

//...
.mountServiceHttpApp(
  orMyHandler(httpRoutes0),
  "/"
)
// ...

As an aside, I've never seen someone mount a lot of services onto one builder like this. I'd rewrite your example like this instead:

JettyBuilder[IO]
  .bindAny()
  .mountServiceHttpApp(
    myOrHandler(
      Router(
        "" -> httpRoutes0,
        "prefix1" -> httpRoutes1,
        ...
        "prefixN" -> httpRoutesN
      )
    ),
    "/"
  )

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this is exactly what I have been looking for !

More generally, do you think there is a need for a JettyBuilder.mountServiceHttpApp ? I would say yes because as .mountServiceHttpApp expects a total function then it is clearer to see how NotFound for instance are handled. Also it would make sense to unify the APIs between Jetty and Blaze. What do you think ?

Copy link
Contributor

@nigredo-tori nigredo-tori Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added clarity is not as important here; AFAICT, people tend to separate their routing logic from the server itself (http4s is great in that), so whoever wants to be sure they have matched all the possible options is probably already using HttpApp anyway, and just rewraps that as HttpRoutes. I think mountServiceHttpApp (or, rather, mountHttpApp) will be useful chiefly because it would allow us to avoid that annoying rewrapping.

Also it would make sense to unify the APIs between Jetty and Blaze.

Not really. As it was mentioned here, Jetty and Blaze builders have fundamentally different capabilities for mounting routes, so I don't see an attempt to unify them leading to anything other than extra complexity and confusion.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think mountServiceHttpApp (or, rather, mountHttpApp) will be useful chiefly because it would allow us to avoid that annoying rewrapping.

Avoiding useless rewrapping on the library-user side is always good, I will simplify this PR and follow your advices

Copy link
Contributor

@nigredo-tori nigredo-tori left a comment

👍 once these two minor things are addressed.

@@ -14,7 +14,7 @@ import scala.concurrent.duration._

class BlockingHttp4sServletSpec extends Http4sSpec {

lazy val service = HttpRoutes.of[IO] {
lazy val service = HttpApp[IO] {
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This match is now non-exhaustive! Please add .orNotFound below.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was fixed in my latest commit

import scala.io.Source
import scala.util.Try

class NotFoundHttpAppSpec extends Http4sSpec {
Copy link
Contributor

@nigredo-tori nigredo-tori Dec 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer need this spec - it doesn't test anything that isn't covered elsewhere.

Copy link
Contributor Author

@CharlesD91 CharlesD91 Dec 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I removed it

@CharlesD91
Copy link
Contributor Author

@CharlesD91 CharlesD91 commented Dec 18, 2019

Hello @nigredo-tori , I fixed the PR following your advice.


For some reason Travis CI keeps failing on 1 of the 3 test machines:

[error] ## Exception when compiling 2 sources to /home/travis/build/http4s/http4s/json4s-jackson/target/scala-2.13/classes
[error] scala.reflect.internal.FatalError: Error accessing /home/travis/.cache/coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.9.8/jackson-databind-2.9.8.jar 

but it is successful on the 2 others.

Please let me know what you think and thank you for your help !

@nigredo-tori
Copy link
Contributor

@nigredo-tori nigredo-tori commented Dec 18, 2019

I kept NotFoundHttpAppSpec because I think it is good to have a test on JettyBuilder.mountHttpApp.

We already test it in JettyServerSpec since mountService factors through it.

Also it displays how NotFound can be handled, while it is handled by default when using JettyBuilder.mountService.

This by itself is not a sufficient reason for adding tests. Also, the most common case for making a HttpApp from HttpRoutes (.orNotFound) is already used extensively throughout the documentation. Manually handling this depends on the use case, and should be reasonably easy to arrive to for people familiar with Kleisli and OptionT, just by looking at HttpRoutes and HttpApp types.

@nigredo-tori
Copy link
Contributor

@nigredo-tori nigredo-tori commented Dec 18, 2019

👍 from me. @hamnis, @rossabaker, please take a look again - this PR has gotten a lot lighter.

hamnis
hamnis approved these changes Dec 19, 2019
@hamnis
Copy link
Contributor

@hamnis hamnis commented Dec 19, 2019

I cleared the travis cache and restarted the build, can be merged on build success.

@hamnis hamnis merged commit f1204a2 into http4s:master Dec 20, 2019
2 checks passed
@CharlesD91 CharlesD91 mentioned this pull request Dec 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants