From 0c549c6b4d3af964df7bc908320db37c5fe10ea7 Mon Sep 17 00:00:00 2001 From: Juan Pedro Moreno Date: Thu, 18 Jan 2018 12:12:17 +0100 Subject: [PATCH] Updates Docs regarding Metrics Reporting (#145) * Updates docs * Small lang updates * Clarifying artifact table * Fixes tut --- README.md | 75 ++++++++++------ docs/src/main/tut/README.md | 174 ++++++++++++++++++++++++++++++++---- 2 files changed, 208 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index d084c42a9..12d04e8fd 100644 --- a/README.md +++ b/README.md @@ -7,56 +7,79 @@ # freestyle-rpc -Freestyle RPC is a purely functional library for building [RPC] endpoint based services with support for [RPC] and [HTTP/2]. +Freestyle RPC is a purely functional library for building [RPC] endpoint-based services with support for [RPC] and [HTTP/2]. -Also known as [frees-rpc], it brings the ability to combine [RPC] protocols, services and clients in your `Freestyle` program, thanks to [gRPC]. +Also known as [frees-rpc], it brings the ability to combine [RPC] protocols, services, and clients in your `Freestyle` program, thanks to [gRPC]. ## Installation -`frees-rpc` is cross-built for Scala `2.11.x` and `2.12.x`: +`frees-rpc` is cross-built for Scala `2.11.x` and `2.12.x`. + +It's divided into multiple and different artifacts, grouped by scope: + +* `Server`: specifically for the RPC server. +* `Client`: focused on the RPC auto-derived clients by `frees-rpc`. +* `Server/Client`: used from other artifacts for both Server and Client. +* `Test`: useful to test `frees-rpc` applications. + +*Artifact Name* | *Scope* | *Mandatory* | *Description* +--- | --- | --- | --- +`frees-rpc-server` | Server | Yes | Needed to attach RPC Services and spin-up an RPC Server. +`frees-rpc-client-core` | Client | Yes | Mandatory to define protocols and auto-derived clients. +`frees-rpc-client-netty` | Client | Yes* | Optional if you use `OkHttp`, required from the client perspective. +`frees-rpc-client-okhttp` | Client | Yes* | Optional if you use `Netty`, required from the client perspective. +`frees-rpc-config` | Server/Client | No | It provides configuration helpers using [frees-config] to load the application configuration values. +`frees-rpc-prometheus-server` | Server | No | Scala interceptors which can be used to monitor gRPC services using Prometheus, on the _Server_ side. +`frees-rpc-prometheus-client` | Client | No | Scala interceptors which can be used to monitor gRPC services using Prometheus, on the _Client_ side. +`frees-rpc-prometheus-shared` | Server/Client | No | Common code for both the client and the server in the prometheus scope. +`frees-rpc-dropwizard-server` | Server | No | Scala interceptors which can be used to monitor gRPC services using Dropwizard metrics, on the _Server_ side. +`frees-rpc-dropwizard-client` | Client | No | Scala interceptors which can be used to monitor gRPC services using Dropwizard metrics, on the _Client_ side. +`frees-rpc-interceptors` | Server/Client | No | Commons related to gRPC interceptors. +`frees-rpc-testing` | Test | No | Utilities to test out `frees-rpc` applications. It provides the `grpc-testing` library as the transitive dependency. +`frees-rpc-common` | Server/Client | Provided* | Common things that are used throughout the project. +`frees-rpc-internal` | Server/Client | Provided* | Macros. +`frees-rpc-async` | Server/Client | Provided* | Async instances useful for interacting with the RPC services on both sides, server and the client. + +* `Yes*`: on the client-side, you must choose either `Netty` or `OkHttp` as the transport layer. +* `Provided*`: you don't need to add it to your build, it'll be transitively provided when using other dependencies. + +You can install any of these dependencies in your build as follows: [comment]: # (Start Replace) ```scala // required for the RPC Server: -libraryDependencies += "io.frees" %% "frees-rpc-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-server" % "0.9.0" // required for a protocol definition: -libraryDependencies += "io.frees" %% "frees-rpc-client-core" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-core" % "0.9.0" // required for the use of the derived RPC Client/s, using either Netty or OkHttp as transport layer: -libraryDependencies += "io.frees" %% "frees-rpc-client-netty" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-netty" % "0.9.0" // or: -libraryDependencies += "io.frees" %% "frees-rpc-client-okhttp" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-okhttp" % "0.9.0" // optional - for both server and client configuration. -libraryDependencies += "io.frees" %% "frees-rpc-config" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-config" % "0.9.0" + +// optional - for both server and client metrics reporting, using Prometheus. +libraryDependencies += "io.frees" %% "frees-rpc-prometheus-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-prometheus-client" % "0.9.0" + +// optional - for both server and client metrics reporting, using Dropwizard. +libraryDependencies += "io.frees" %% "frees-rpc-dropwizard-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-dropwizard-client" % "0.9.0" ``` [comment]: # (End Replace) -Note: `frees-rpc-config` provides some configuration helpers using [frees-config] to load the application configuration values. - ## Documentation -The full documentation is available at [frees-rpc](http://frees.io/docs/rpc) site. - -## Sbt Modules - -`frees-rpc` code is placed in different sbt modules: - -* `frees-rpc-common`: contains the protocol types, with the minimum set of dependencies. -* `frees-rpc-async`: contains just the async implicit instances (NTs between effect/async types). -* `frees-rpc-internal` where the macros are placed. -* `frees-rpc-client-core`: algebra and code related to the RPC clients. -* `frees-rpc-client-netty`: it doesn't add any additional code, just a transport layer provider based on `grpc-netty`. -* `frees-rpc-client-okhttp`: similar to the `Netty` one, it doesn't add any additional code, just a transport layer provider based on `grpc-okhttp`. -* `frees-rpc-server`: algebra and code related to the RPC server. -* `frees-rpc-config`: helpers to be able to load the client and the server configuration. +The full documentation is available at the [frees-rpc](http://frees.io/docs/rpc) site. ## Demo -See [freestyle-rpc-examples](https://github.com/frees-io/freestyle-rpc-examples) repo. +See the [freestyle-rpc-examples](https://github.com/frees-io/freestyle-rpc-examples) repo. [RPC]: https://en.wikipedia.org/wiki/Remote_procedure_call [HTTP/2]: https://http2.github.io/ @@ -69,6 +92,6 @@ See [freestyle-rpc-examples](https://github.com/frees-io/freestyle-rpc-examples) Freestyle is designed and developed by 47 Degrees -Copyright (C) 2017-2018 47 Degrees. +Copyright (C) 2017-2018 47 Degrees. [comment]: # (End Copyright) \ No newline at end of file diff --git a/docs/src/main/tut/README.md b/docs/src/main/tut/README.md index b9f1e11cd..5d1cc32f3 100644 --- a/docs/src/main/tut/README.md +++ b/docs/src/main/tut/README.md @@ -8,7 +8,7 @@ permalink: /docs/rpc [RPC] atop **Freestyle** is **`frees-rpc`**. -Freestyle RPC is a purely functional library for building RPC endpoint based services with support for [RPC] and [HTTP/2]. +Freestyle RPC is a purely functional library for building RPC endpoint-based services with support for [RPC] and [HTTP/2]. @@ -37,6 +37,10 @@ Freestyle RPC is a purely functional library for building RPC endpoint based ser - [Runtime Implicits](#runtime-implicits-1) - [Client Program](#client-program) - [frees-rpc annotations](#frees-rpc-annotations) + - [Metrics Reporting](#metrics-reporting) + - [Monitor Server Calls](#monitor-server-calls) + - [Monitor Client Calls](#monitor-client-calls) + - [Dropwizard Metrics](#dropwizard-metrics) - [Next Steps](#next-steps) - [Comparing HTTP and RPC](#comparing-http-and-rpc) - [References](#references) @@ -49,30 +53,66 @@ Freestyle RPC is a purely functional library for building RPC endpoint based ser ## Installation -`frees-rpc` is cross-built for Scala `2.11.x` and `2.12.x`: +`frees-rpc` is cross-built for Scala `2.11.x` and `2.12.x`. + +It's divided into multiple and different artifacts, grouped by scope: + +* `Server`: specifically for the RPC server. +* `Client`: focused on the RPC auto-derived clients by `frees-rpc`. +* `Server/Client`: used from other artifacts for both Server and Client. +* `Test`: useful to test `frees-rpc` applications. + +*Artifact Name* | *Scope* | *Mandatory* | *Description* +--- | --- | --- | --- +`frees-rpc-server` | Server | Yes | Needed to attach RPC Services and spin-up an RPC Server. +`frees-rpc-client-core` | Client | Yes | Mandatory to define protocols and auto-derived clients. +`frees-rpc-client-netty` | Client | Yes* | Optional if you use `OkHttp`, required from the client perspective. +`frees-rpc-client-okhttp` | Client | Yes* | Optional if you use `Netty`, required from the client perspective. +`frees-rpc-config` | Server/Client | No | It provides configuration helpers using [frees-config] to load the application configuration values. +`frees-rpc-prometheus-server` | Server | No | Scala interceptors which can be used to monitor gRPC services using Prometheus, on the _Server_ side. +`frees-rpc-prometheus-client` | Client | No | Scala interceptors which can be used to monitor gRPC services using Prometheus, on the _Client_ side. +`frees-rpc-prometheus-shared` | Server/Client | No | Common code for both the client and the server in the prometheus scope. +`frees-rpc-dropwizard-server` | Server | No | Scala interceptors which can be used to monitor gRPC services using Dropwizard metrics, on the _Server_ side. +`frees-rpc-dropwizard-client` | Client | No | Scala interceptors which can be used to monitor gRPC services using Dropwizard metrics, on the _Client_ side. +`frees-rpc-interceptors` | Server/Client | No | Commons related to gRPC interceptors. +`frees-rpc-testing` | Test | No | Utilities to test out `frees-rpc` applications. It provides the `grpc-testing` library as the transitive dependency. +`frees-rpc-common` | Server/Client | Provided* | Common things that are used throughout the project. +`frees-rpc-internal` | Server/Client | Provided* | Macros. +`frees-rpc-async` | Server/Client | Provided* | Async instances useful for interacting with the RPC services on both sides, server and the client. + +* `Yes*`: on the client-side, you must choose either `Netty` or `OkHttp` as the transport layer. +* `Provided*`: you don't need to add it to your build, it'll be transitively provided when using other dependencies. + +You can install any of these dependencies in your build as follows: [comment]: # (Start Replace) ```scala // required for the RPC Server: -libraryDependencies += "io.frees" %% "frees-rpc-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-server" % "0.9.0" // required for a protocol definition: -libraryDependencies += "io.frees" %% "frees-rpc-client-core" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-core" % "0.9.0" // required for the use of the derived RPC Client/s, using either Netty or OkHttp as transport layer: -libraryDependencies += "io.frees" %% "frees-rpc-client-netty" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-netty" % "0.9.0" // or: -libraryDependencies += "io.frees" %% "frees-rpc-client-okhttp" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-client-okhttp" % "0.9.0" // optional - for both server and client configuration. -libraryDependencies += "io.frees" %% "frees-rpc-config" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-config" % "0.9.0" + +// optional - for both server and client metrics reporting, using Prometheus. +libraryDependencies += "io.frees" %% "frees-rpc-prometheus-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-prometheus-client" % "0.9.0" + +// optional - for both server and client metrics reporting, using Dropwizard. +libraryDependencies += "io.frees" %% "frees-rpc-dropwizard-server" % "0.9.0" +libraryDependencies += "io.frees" %% "frees-rpc-dropwizard-client" % "0.9.0" ``` [comment]: # (End Replace) -Note: `frees-rpc-config` provides some configuration helpers using [frees-config] to load the application configuration values. - ## About gRPC > [gRPC](https://grpc.io/about/) is a modern, open source, and high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking, and authentication. It's also applicable in the last mile of distributed computing to connect devices, mobile applications, and browsers to backend services. @@ -247,7 +287,7 @@ object protocol { } } ``` - + We are also using some additional annotations: * `@option`: used to define the equivalent headers in `.proto` files. @@ -608,19 +648,18 @@ object gclient { trait Implicits extends CommonRuntime { val channelFor: ManagedChannelFor = - ConfigForAddress[ChannelConfig.Op]("rpc.host", "rpc.port") - .interpret[Try] match { + ConfigForAddress[Try]("rpc.host", "rpc.port") match { case Success(c) => c case Failure(e) => e.printStackTrace() throw new RuntimeException("Unable to load the client configuration", e) + } implicit val serviceClient: Greeter.Client[Task] = Greeter.client[Task](channelFor) } object implicits extends Implicits - } ``` @@ -638,11 +677,14 @@ import service._ import gclient.implicits._ import monix.eval.Task +import scala.concurrent.Await +import scala.concurrent.duration.Duration + object RPCDemoApp { def main(args: Array[String]): Unit = { - val result: HelloResponse = serviceClient.sayHello(HelloRequest("foo")).unsafeRunSync + val result = Await.result(serviceClient.sayHello(HelloRequest("foo")).runAsync, Duration.Inf) println(s"Result = $result") @@ -663,6 +705,108 @@ Annotation | Scope | Arguments | Description @message | `Case Class` | - | Tags a case class a protobuf message. @option | `Object` | [name: String, value: String, quote: Boolean] | used to define the equivalent headers in `.proto` files +## Metrics Reporting + +[frees-rpc] currently provides two different ways to monitor [gRPC] services: `Prometheus` and `Dropwizard` (using the `Prometheus` extension). The usage is quite similar for both. + +### Monitor Server Calls + +In order to monitor the RPC calls on the server side, it's necessary to intercept them. We'll see how to do this in the next code fragment: + +```tut:silent +import cats.~> +import cats.effect.IO +import freestyle.rpc.server._ +import freestyle.rpc.server.handlers._ +import freestyle.rpc.server.implicits._ +import freestyle.async.catsEffect.implicits._ +import service._ + +import io.prometheus.client.CollectorRegistry +import freestyle.rpc.prometheus.shared.Configuration +import freestyle.rpc.prometheus.server.MonitoringServerInterceptor + +object InterceptingServerCalls extends CommonRuntime { + + import freestyle.rpc.interceptors.implicits._ + + lazy val cr: CollectorRegistry = new CollectorRegistry() + lazy val monitorInterceptor = MonitoringServerInterceptor( + Configuration.defaultBasicMetrics.withCollectorRegistry(cr) + ) + + implicit val greeterServiceHandler: ServiceHandler[IO] = new ServiceHandler[IO] + + val grpcConfigs: List[GrpcConfig] = List( + AddService(Greeter.bindService[IO].interceptWith(monitorInterceptor)) + ) + + implicit val serverW: ServerW = ServerW(8080, grpcConfigs) + +} +``` + +### Monitor Client Calls + +In this case, in order to intercept the client calls we need additional configuration settings (by using `AddInterceptor`): + +```tut:silent +import cats.implicits._ +import cats.effect.IO +import freestyle.free.config.implicits._ +import freestyle.async.catsEffect.implicits._ +import freestyle.rpc.client._ +import freestyle.rpc.client.config._ +import freestyle.rpc.client.implicits._ +import monix.eval.Task +import io.grpc.ManagedChannel +import service._ + +import scala.util.{Failure, Success, Try} + +import freestyle.rpc.prometheus.shared.Configuration +import freestyle.rpc.prometheus.client.MonitoringClientInterceptor + +object InterceptingClientCalls extends CommonRuntime { + + val channelFor: ManagedChannelFor = + ConfigForAddress[Try]("rpc.host", "rpc.port") match { + case Success(c) => c + case Failure(e) => + e.printStackTrace() + throw new RuntimeException("Unable to load the client configuration", e) + } + + implicit val serviceClient: Greeter.Client[Task] = + Greeter.client[Task]( + channelFor = channelFor, + channelConfigList = List( + UsePlaintext(true), + AddInterceptor( + MonitoringClientInterceptor( + Configuration.defaultBasicMetrics + ) + ) + ) + ) +} +``` + +That's using `Prometheus` to monitor both [gRPC] ends. + +### Dropwizard Metrics + +The usage is equivalent, however, in this case, we need to put an instance of `com.codahale.metrics.MetricRegistry` on the scene, then, using the _Dropwizard_ integration that _Prometheus_ already provides (`DropwizardExports`) you can associate it with the collector registry: + +```tut:silent +import com.codahale.metrics.MetricRegistry +import io.prometheus.client.dropwizard.DropwizardExports + +val metrics: MetricRegistry = new MetricRegistry +val configuration: Configuration = Configuration.defaultBasicMetrics +configuration.collectorRegistry.register(new DropwizardExports(metrics)) +``` + ## Next Steps If you want to delve deeper into [frees-rpc], we have a complete example at the [freestyle-rpc-examples] repository, which is based on the [Route Guide Demo](https://grpc.io/docs/tutorials/basic/java.html#generating-client-and-server-code) originally shared by the [gRPC Java Project](https://github.com/grpc/grpc-java/tree/6ea2b8aacb0a193ac727e061bc228b40121460e3/examples/src/main/java/io/grpc/examples/routeguide). @@ -701,4 +845,4 @@ This extra section is not specifically about [frees-rpc]. Very often our microse [cats-effect]: https://github.com/typelevel/cats-effect [freestyle-rpc-examples]: https://github.com/frees-io/freestyle-rpc-examples [Metrifier]: https://github.com/47deg/metrifier -[frees-config]: http://frees.io/docs/patterns/config/ \ No newline at end of file +[frees-config]: http://frees.io/docs/patterns/config/