Integration with ZIO metrics (#801)

* Get rid of Baggage.logAnnotated layer constructor

* Seamless integration with ZIO and JVM metrics

* scalafix

* Update docs and scala-cli regarding updated OpenTelemetry.metrics

* Support for ZIO metrics Frequency

* scalafmt

* Limit access scope for internal stuff

* Update README

* Add a TODO note to implement updateSummary method

* Polish some docs

docs/opentelemetry-example.md

@@ -9,7 +9,9 @@ For an explanation in more detail, check the [OpenTracing Example](opentracing-e
 
 We're going to show an example of how to pass contextual information using [Baggage](https://opentelemetry.io/docs/concepts/signals/baggage/) and collect traces, metrics, and logs.
 
----
+# Run
+
+## Print OTEL signals to console
 
 By default the example code uses [OTLP Logging Exporters](https://github.com/open-telemetry/opentelemetry-java/tree/main/exporters/logging-otlp) to print all signals to stdout in OTLP JSON encoding. This means that you can run the application immediately and observe the results.
 
@@ -28,11 +30,11 @@ Now perform the following request to see the results immediately:
 curl -X GET http://localhost:8080/statuses
 ```
 
----
+## Publish OTEL signals to other observability platforms
 
 In case you want to try different observability platforms such as [Jaeger](https://www.jaegertracing.io/), [Fluentbit](https://fluentbit.io/), [Seq](https://datalust.co/seq), [DataDog](https://www.datadoghq.com/), [Honeycomb](https://www.honeycomb.io/) or others, please change the [OtelSdk.scala](https://github.com/zio/zio-telemetry/blob/series/2.x/opentelemetry-example/src/main/scala/zio/telemetry/opentelemetry/example/otel/OtelSdk.scala) file by choosing from the available tracer, meter, and logger providers or by implementing your own. 
 
---- 
+## Publish OTEL signals to Jaeger and Seq
 
 We chose [Jaeger](https://www.jaegertracing.io/) for distributed traces and [Seq](https://datalust.co/seq) to store logs to demonstrate how the library works with available open-source observability platforms.
 
@@ -60,4 +62,4 @@ docker run \
   datalust/seq
 ```
 
-Run the application and fire a curl request as shown above, and then head over to [Jaeger UI](http://localhost:16686/) and [Seq UI](http://localhost:80/) to see the result.
+Run the application and fire a curl request as shown above. Head over to [Jaeger UI](http://localhost:16686/) and [Seq UI](http://localhost:80/) to see the result.

docs/opentelemetry.md

@@ -12,7 +12,7 @@ Some of the key features:
 - **ZIO native** - Pleasant API that leverages native ZIO features, such as [Resource Management](https://zio.dev/reference/resource/), [Depenency Injection](https://zio.dev/reference/di/), [Streaming](https://zio.dev/reference/stream/), [Logging](https://zio.dev/reference/observability/logging), [Metrics](https://zio.dev/reference/observability/metrics/), and [ZIO Aspect](https://zio.dev/reference/core/zio/#zio-aspect)
 - **OpenTelemetry Java SDK and ZIO Runtime interoperability** - Protecting users from directly engaging in OTEL context manipulations, offering a straightforward and clear interface for instrumenting spans, metrics, logs, and baggage. In this scenario, the ZIO effect serves as the span's scope.
 - **Seamless signals correlation** -  Automatically correlates spans, metrics, and logs with a surrounding span.
-- **Integration with ZIO capabilities** - Propagation of log annotations, metrics, and other data from the ZIO runtime as OTEL attributes and metrics.
+- **Integration with ZIO capabilities** - Propagation of log annotations, metrics, and other data from the ZIO runtime as OTEL attributes and metric signals.
 
 ## Installation
 
@@ -36,6 +36,51 @@ For the complete list of available Java artifacts, please consult the informatio
 
 All examples below can be run using amazing [Scala CLI](https://scala-cli.virtuslab.org/). You can find their full copies in the `scala-cli/opentelemetry/` directory. To run, type `scala-cli <AppName>.scala` while in the directory where the file is located.
 
+### Setup
+
+The `zio.telemetry.opentelemetry.OpenTelemetry` (aka entry point) offers a comprehensive set of layers for instrumenting your ZIO application. 
+
+First of all, you need to provide an instance of `io.opentelemetry.api.Opentelemetry`.
+In case you don't need an automatic instrumentation, you can use `OpenTelemetry.custom` layer. It receives a scoped ZIO effect indicating that the provided instance will be closed when the application is shut down. Here is an example:
+
+```scala
+import zio._
+import zio.telemetry.opentelemetry.OpenTelemetry
+import io.opentelemetry.sdk.OpenTelemetrySdk
+import io.opentelemetry.api
+
+def custom(resourceName: String): TaskLayer[api.OpenTelemetry] =
+  OpenTelemetry.custom(
+    for {
+      tracerProvider <- TracerProvider.stdout(resourceName)
+      meterProvider  <- MeterProvider.stdout(resourceName)
+      loggerProvider <- LoggerProvider.stdout(resourceName)
+      openTelemetry  <- ZIO.fromAutoCloseable(
+                          ZIO.succeed(
+                            OpenTelemetrySdk
+                              .builder()
+                              .setTracerProvider(tracerProvider)
+                              .setMeterProvider(meterProvider)
+                              .setLoggerProvider(loggerProvider)
+                              .build
+                          )
+                        )
+    } yield openTelemetry
+  )
+```
+
+The library depends only on `opentelemetry-api` which means you have to manage an initialization of providers and depenendencies for `opentelemetry-sdk`, and `opentelemetry-exporter-*` inside your application.
+
+For more details, please have a look at the source code of the [example application](https://github.com/zio/zio-telemetry/tree/series/2.x/opentelemetry-example/src/main/scala/zio/telemetry/opentelemetry/example).
+
+#### Usage with OpenTelemetry automatic instrumentation
+
+OpenTelemetry provides a [JVM agent for automatic instrumentation](https://opentelemetry.io/docs/instrumentation/java/automatic/) which supports many [popular Java libraries](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md).
+Since [version 1.25.0](https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v1.25.0) OpenTelemetry JVM agent supports ZIO.
+
+To enable interoperability between automatic instrumentation and `zio-opentelemetry`, `Tracing` has to be created
+using `ContextStorage` backed by OpenTelemetry's native `Context` and `Tracer` provided by globally registered `TracerProvider`. It means that instead of `ContextStorage.fiberRef` and `OpenTelemetry.custom` you have to provide `ContextStorage.native` and `OpenTelemetry.global` layers.
+
 ### Tracing
 
 To send [Trace signals](https://opentelemetry.io/docs/concepts/signals/traces/), you will need a `Tracing` service in your environment. For this, use the `OpenTelemetry.tracing` layer which in turn requires an instance of `OpenTelemetry` provided by Java SDK and a suitable `ContextStorage` implementation. The `Tracing` API includes methods for creating spans, as well as for adding attributes and events to them. 
@@ -145,12 +190,12 @@ To send [Metric signals](https://opentelemetry.io/docs/concepts/signals/metrics/
 As a rule of thumb, observable instruments must be initialized on an application startup. They are scoped, so you should not be worried about shutting them down manually.
 
 ```scala
-//> using scala "2.13.12"
-//> using dep dev.zio::zio:2.0.20
-//> using dep dev.zio::zio-opentelemetry:3.0.0-RC20
-//> using dep io.opentelemetry:opentelemetry-sdk:1.33.0
-//> using dep io.opentelemetry:opentelemetry-sdk-trace:1.33.0
-//> using dep io.opentelemetry:opentelemetry-exporter-logging-otlp:1.33.0
+//> using scala "2.13.13"
+//> using dep dev.zio::zio:2.0.21
+//> using dep dev.zio::zio-opentelemetry:3.0.0-RC22
+//> using dep io.opentelemetry:opentelemetry-sdk:1.36.0
+//> using dep io.opentelemetry:opentelemetry-sdk-trace:1.36.0
+//> using dep io.opentelemetry:opentelemetry-exporter-logging-otlp:1.36.0
 //> using dep io.opentelemetry.semconv:opentelemetry-semconv:1.22.0-alpha
 
 import io.opentelemetry.sdk.trace.SdkTracerProvider
@@ -278,7 +323,7 @@ object MetricsApp extends ZIOAppDefault {
       .provide(
         otelSdkLayer,
         ContextStorage.fiberRef,
-        OpenTelemetry.meter(instrumentationScopeName),
+        OpenTelemetry.metrics(instrumentationScopeName),
         OpenTelemetry.tracing(instrumentationScopeName),
         tickCounterLayer,
         tickRefLayer
@@ -287,6 +332,10 @@ object MetricsApp extends ZIOAppDefault {
 }
 ```
 
+#### Integration with ZIO metrics
+
+To enable seamless integration with [ZIO metrics](https://zio.dev/reference/observability/metrics/), use the `OpenTelemetry.zioMetrics` layer. If you also need to publish JVM metrics, be sure to include `DefaultJvmMetrics.live.unit`.
+
 ### Logging
 
 To send [Log signals](https://opentelemetry.io/docs/concepts/signals/logs/), you will need a `Logging` service in your environment. For this, use the `OpenTelemetry.logging` layer which in turn requires an instance of `OpenTelemetry` provided by Java SDK and a suitable `ContextStorage` implementation. You can achieve the same by incorporating [Logger MDC auto-instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/logger-mdc-instrumentation.md), so the rule of thumb is to use the `Logging` service when you need to propagate ZIO log annotations as log record attributes or, for some reason you don't want to use auto-instrumentation.
@@ -568,11 +617,3 @@ object PropagatingApp extends ZIOAppDefault {
 
 }
 ```
-
-### Usage with OpenTelemetry automatic instrumentation
-
-OpenTelemetry provides a [JVM agent for automatic instrumentation](https://opentelemetry.io/docs/instrumentation/java/automatic/) which supports many [popular Java libraries](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md).
-Since [version 1.25.0](https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v1.25.0) OpenTelemetry JVM agent supports ZIO.
-
-To enable interoperability between automatic instrumentation and `zio-opentelemetry`, `Tracing` has to be created
-using `ContextStorage` backed by OpenTelemetry's native `Context` and `Tracer` provided by globally registered `TracerProvider`. It means that instead of `ContextStorage.fiberRef` and `OpenTelemetry.custom` you have to provide `ContextStorage.native` and `OpenTelemetry.global` layers.

opentelemetry-example/src/main/scala/zio/telemetry/opentelemetry/example/BackendApp.scala

@@ -9,6 +9,7 @@ import zio.telemetry.opentelemetry.context.ContextStorage
 import zio.telemetry.opentelemetry.OpenTelemetry
 import zio.telemetry.opentelemetry.example.otel.OtelSdk
 import zio.telemetry.opentelemetry.metrics.Meter
+import zio.metrics.jvm.DefaultJvmMetrics
 
 object BackendApp extends ZIOAppDefault {
 
@@ -51,12 +52,14 @@ object BackendApp extends ZIOAppDefault {
         BackendHttpApp.live,
         OtelSdk.custom(resourceName),
         OpenTelemetry.tracing(instrumentationScopeName),
-        OpenTelemetry.meter(instrumentationScopeName),
+        OpenTelemetry.metrics(instrumentationScopeName),
         OpenTelemetry.logging(instrumentationScopeName),
         OpenTelemetry.baggage(),
+        OpenTelemetry.zioMetrics,
+        DefaultJvmMetrics.live.unit,
         globalTickCounterLayer,
         tickRefLayer,
-        ContextStorage.fiberRef
+        ContextStorage.fiberRef,
       )
 
 }

opentelemetry/src/main/scala/zio/telemetry/opentelemetry/OpenTelemetry.scala

@@ -2,10 +2,12 @@ package zio.telemetry.opentelemetry
 
 import io.opentelemetry.api
 import zio._
+import zio.metrics.{MetricClient, MetricListener}
 import zio.telemetry.opentelemetry.baggage.Baggage
 import zio.telemetry.opentelemetry.context.ContextStorage
 import zio.telemetry.opentelemetry.logging.Logging
 import zio.telemetry.opentelemetry.metrics.Meter
+import zio.telemetry.opentelemetry.metrics.internal.{Instrument, InstrumentRegistry, OtelMetricListener}
 import zio.telemetry.opentelemetry.tracing.Tracing
 
 /**
@@ -82,12 +84,12 @@ object OpenTelemetry {
    * @param schemaUrl
    *   schema URL
    */
-  def meter(
+  def metrics(
     instrumentationScopeName: String,
     instrumentationVersion: Option[String] = None,
     schemaUrl: Option[String] = None
-  ): URLayer[api.OpenTelemetry with ContextStorage, Meter] = {
-    val meterLayer = ZLayer(
+  ): URLayer[api.OpenTelemetry with ContextStorage, Meter with Instrument.Builder] = {
+    val meterLayer   = ZLayer(
       ZIO.serviceWith[api.OpenTelemetry] { openTelemetry =>
         val builder = openTelemetry.meterBuilder(instrumentationScopeName)
 
@@ -97,8 +99,32 @@ object OpenTelemetry {
         builder.build()
       }
     )
+    val builderLayer = meterLayer >>> Instrument.Builder.live
 
-    meterLayer >>> Meter.live
+    builderLayer >+> (builderLayer >>> Meter.live)
+  }
+
+  /**
+   * Use when you want to allow a seamless integration with ZIO runtime and JVM metrics
+   *
+   * By default this layer enables the propagation of ZIO runtime metrics only. For JVM metrics you need to provide
+   * `DefaultJvmMetrics.live.unit`.
+   */
+  def zioMetrics: URLayer[Instrument.Builder, Unit] = {
+    val metricListenerLifecycleLayer = ZLayer.scoped {
+      ZIO.serviceWithZIO[MetricListener] { metricListener =>
+        Unsafe.unsafe { implicit unsafe =>
+          ZIO.acquireRelease(
+            ZIO.succeed(MetricClient.addListener(metricListener))
+          )(_ => ZIO.succeed(MetricClient.removeListener(metricListener)))
+        }
+      }
+    }
+
+    Runtime.enableRuntimeMetrics >>>
+      InstrumentRegistry.concurrent >>>
+      OtelMetricListener.zioMetrics >>>
+      metricListenerLifecycleLayer
   }
 
   /**

opentelemetry/src/main/scala/zio/telemetry/opentelemetry/baggage/Baggage.scala

@@ -199,7 +199,4 @@ object Baggage {
       }
     }
 
-  def logAnnotated: URLayer[ContextStorage, Baggage] =
-    live(logAnnotated = true)
-
 }

opentelemetry/src/main/scala/zio/telemetry/opentelemetry/context/ContextStorage.scala

@@ -92,8 +92,8 @@ object ContextStorage {
     ZLayer.scoped(
       FiberRef
         .make[Context](Context.root())
-        .flatMap { ref =>
-          ZIO.succeed(new ZIOFiberRef(ref))
+        .map { ref =>
+          new ZIOFiberRef(ref)
         }
     )
 

opentelemetry/src/main/scala/zio/telemetry/opentelemetry/metrics/Counter.scala

@@ -2,16 +2,18 @@ package zio.telemetry.opentelemetry.metrics
 
 import io.opentelemetry.api.common.Attributes
 import io.opentelemetry.api.metrics.LongCounter
+import io.opentelemetry.context.Context
 import zio._
 import zio.telemetry.opentelemetry.context.ContextStorage
+import zio.telemetry.opentelemetry.metrics.internal.Instrument
 
 /**
  * A Counter instrument that records values of type `A`
  *
  * @tparam A
  *   according to the specification, it can be either [[scala.Long]] or [[scala.Double]] type
  */
-trait Counter[-A] {
+trait Counter[-A] extends Instrument[A] {
 
   /**
    * Records a value.
@@ -43,8 +45,11 @@ object Counter {
   private[metrics] def long(counter: LongCounter, ctxStorage: ContextStorage): Counter[Long] =
     new Counter[Long] {
 
+      override def record0(value: Long, attributes: Attributes = Attributes.empty, context: Context): Unit =
+        counter.add(value, attributes, context)
+
       override def add(value: Long, attributes: Attributes = Attributes.empty)(implicit trace: Trace): UIO[Unit] =
-        ctxStorage.get.map(counter.add(value, attributes, _))
+        ctxStorage.get.map(record0(value, attributes, _))
 
       override def inc(attributes: Attributes = Attributes.empty)(implicit trace: Trace): UIO[Unit] =
         add(1L, attributes)

opentelemetry/src/main/scala/zio/telemetry/opentelemetry/metrics/Histogram.scala

@@ -2,16 +2,18 @@ package zio.telemetry.opentelemetry.metrics
 
 import io.opentelemetry.api.common.Attributes
 import io.opentelemetry.api.metrics.DoubleHistogram
+import io.opentelemetry.context.Context
 import zio._
 import zio.telemetry.opentelemetry.context.ContextStorage
+import zio.telemetry.opentelemetry.metrics.internal.Instrument
 
 /**
  * A Histogram instrument that records values of type `A`
  *
  * @tparam A
  *   according to the specification, it can be either [[scala.Long]] or [[scala.Double]] type
  */
-trait Histogram[-A] {
+trait Histogram[-A] extends Instrument[A] {
 
   /**
    * Records a value.
@@ -33,8 +35,11 @@ object Histogram {
   private[metrics] def double(histogram: DoubleHistogram, ctxStorage: ContextStorage): Histogram[Double] =
     new Histogram[Double] {
 
+      override def record0(value: Double, attributes: Attributes = Attributes.empty, context: Context): Unit =
+        histogram.record(value, attributes, context)
+
       override def record(value: Double, attributes: Attributes = Attributes.empty)(implicit trace: Trace): UIO[Unit] =
-        ctxStorage.get.map(histogram.record(value, attributes, _))
+        ctxStorage.get.map(record0(value, attributes, _))
 
     }