Best practices and abstractions

[sergeykolbasov]

When we've started to use finch (from v0.5) as a basis for our REST/HTTP API we've struggled with a lack of some practices, guidelines and how-to about abstractions on top of underlying Service[HttpRequest, HttpResponse].

Finch by itself is a pretty low level layer which responds to work with HTTP requests/responses with some neat monadic extensions/ways to do. And nothing more. Finatra, for example, was built around Filter/Controller and its composition.

So we decided to abstract it on our own.

Now our simple CRUD controller looks like that:

class ItemsApiController(implicit val injector: Injector) extends Controller {

  private val itemsService = inject[ItemsService] //We use scaldi as DI
  private val itemReader = body.as[Item]

  def findItemById(itemId: Long): Action = securedAction { reqContext =>
    itemsService findItemById itemId
  }

  def userItems: Action = securedAction(pageReader) { page => implicit reqContext =>
    itemsService.userItems(user.id.get, PageRequest(page))
  }

  def newItem: Action = securedAction(itemReader){ item => implicit reqContext =>
    itemsService.newItem(user.id.get, item)
  }

  def updateItem(itemId: Long): Action = securedAction(itemReader) { item => implicit reqContext =>
    itemsService.updateItem(user.id.get, itemId, item)
  }

  def removeItem(itemId: Long): Action = securedAction { implicit reqContext =>
    itemsService.removeItem(user.id.get, itemId)
  }

  override def routes: Endpoint[HttpRequest, HttpResponse] = {
    (Get    / "api" / "items"        /> userItems)    |
    (Get    / "api" / "items" / long /> findItemById) |
    (Post   / "api" / "items"        /> newItem)      |
    (Put    / "api" / "items" / long /> updateItem)   |
    (Delete / "api" / "items" / long /> removeItem)
  }

}

Looks pretty intuitive and simple as for me.

You may ask why do we ever need a controller layer when it's just a kind of proxy to services?
Well, it's a good place to do lazy authentication, check user permissions, validate input and many other things which are not welcome in a real business logic.
Yet it provides a simple way to compose all controllers together and bind it to HTTP port.

So here is a question.
Do we ever need this bicycle-boilerplate as a part of some finch ecosystem, some simple example & bootstrap for newbies?

I completely understand that this abstraction level should not be a part of finch core just because of its nature. I just wanna know if someone interested in that experience and how should I present it in that case - some article or even finch-subproject.

[travisbrown]

I'll defer to @vkostyukov and others, of course, but my own feeling is that providing higher-level abstractions like this in a Finch subproject would be reasonable—keeping it in this repository would make it easier to stay synced with core, and it would be pretty trivial to move it to a new finch-extensions (or whatever) repository in the Finagle organization if we decide we want to do that someday.

If you're interested in writing a blog post about this work for the Finagle blog, @imliar, let me know—we'd love to have it.

[vkostyukov]

Thanks @imliar!

This is a very very good question. And I believe everybody who is using Finch nowadays faced the same question. We've discussed this at lot, and I remember your participation in early discussions. Here is the short story.

The was a ticket #204 "Finch in Action" about the same question - how to write services (organize your code) with Finch. We tried to come up with something that answers this question and the Micro-thing showed up. Our goal was to hide the HTTP types and be more high-level. But it turned out that it wasn't a good idea since it didn't represent the whole picture, which is very complicated. So it's deprecated right now.

I personally believe that Future Finch with coproduct routers (see #221) merged with request readers into something like Endpoint (see #254) is an answer to your question. Having endpoints, we can say that it's idiomatic to have a bunch of endpoints (representing microservices) that implement your business logic.

// helper/private routines are still endpoints so we can compose everything
val authrorize: Endpoint[AuthUser] = ???
val dumpToLog: Endpoint[Unit] = ??? // side effects

// will be exposed
val getUser: Endpoint[User] = authorize :: dumpToLog :: ....
val postTicket: Endpoint[Ticket] = authorize :: ....

Httpx.serve("8081", getUser :+: postTicket)

That's being said, I hope we can go in that direction and be microservices-oriented thereby merging two layers (services and controllers) into a single layer of microservices that implement "open-closed" principle: open for reuse but closed for modification.

With such toolchain, one can build a system that follow any possible style (i.e., CRUD). And my personal understanding that Finch should stay in library-scope so its users may be innovative in the approaches they use to organize a code around services. But this doesn't mean that everybody should invent his/her own style (backed micro-library) to work with Finch. We will provide a basic schema based on endpoints (see example above) so one can start with it and then wrap them with any number of abstractions (like controllers) he/her feel comfortable with.

Quick example. Here is how the first ever Finch 0.1.0 code looked like. I also asked this question to myself 1 year ago and come up with that silly idea of extendable and composable services. And it worked great for me. As for today, I would probably stay within endpoints mentioned earlier.

The bottom line is I'm doubtful about having this approach merged in Finch. But it might be a good idea to have it as a separate project or at least post a blog about it: this is very very valuable experience that worth sharing with the community.

@imliar can we make a deal? Let's wait until endpoints are shipped (0.8 - 0.9) so you can play with them. I'm quite sure you will be happy to replace two layers of services and controllers with just a single layer or reusable endpoints. If it won't work for you, we will think about CRUD-style extension for Finch exposed as sub-project (maybe finch-crud).

Although, it's not my jurisdiction to stop you from shipping this as a separate project in the Finagle organization. So you can totally work with @travisbrown on this direction.

[vkostyukov]

@imliar Just wanted to make it clear. I'm completely on board with publishing this as a blog post right now and have a link to it from Finch docs. We probably need a new sections in the docs: case studies.

Also it would be nice to have you @imliar in the adopters list :)

[sergeykolbasov]

Okay, guys, I hear you. Let's wait until 0.7 release and I'll write smthing about actual version of finch.

[vkostyukov]

I have an update on this.

[sergeykolbasov]

@vkostyukov OH NOES, 404.

[rpless]

@imliar link: https://gist.github.com/vkostyukov/411a9184f44a136e2ad9

@vkostyukov I like the idea of having micro-frameworks built on top of Finch for those that want them. My primary concern is this line from your write-up:
The main requirement here would be to keep those micro-frameworks as simple as possible.
I'm concerned that they may bloat or that users may want more from the projects and then maintenance becomes an issue. One possible solution is to have them as separate repositories, either as Finagle ecosystem projects or otherwise. I think having them in their own projects would free them of the simplicity restriction and would allow them room to grow if the maintainers felt that was the best direction for that project.
I don't think this concern is a show-stopper for including them in the Finch repo after 1.0, but I think its something to think about before adding new sub-projects.

[vkostyukov]

I was tying to summarize my thoughts on this in Finch's best practices (see Big Picture). TL;DR I wish we could keep Finch in its current state (what I call "vanilla Finch") and not layer anything on top it. Although, because of its pretty generic abstractions, it should be possible to build any kind of face (i.e., CRUD) on top of Finch.

That said, @imliar do you think we can close this ticket for now and for example, publish some Finch docs (under this repo) or even a complete example on CRUD approach?

[sergeykolbasov]

@vkostyukov Yea, I believe we can close it.
I'll try to make some article with a bunch of examples of how we're using finch in our projects with some abstractions over it after 1.0 release with some stable API, just to make sure that it'll be actual for some time.

[vkostyukov]

Thanks @imliar! Feel free to open a PR within a simplified version of CRUD-style controllers - I think it would be super helpful for newcomers. Also, having this example live in the same repo as finch-core will ensure that it's always aligned with the latest API.