Explicit bucket boundaries advice not working with agent SDK

[knbk]

Describe the bug

The advice API to set explicit bucket boundaries was stabilised in v1.32.0, and the opentelemetry-api instrumentation was updated to bridge the stable method to the agent SDK.

Currently the bridging does not work when the agent is used without the SDK incubator extension. The opentelemetry-api-1.31 instrumentation is not loaded because of a muzzle reference mismatch. As a result, the default implementation on the interface is used, which discards the advice.

Steps to reproduce

Use the opentelemetry-api package to create a histogram, and use the setExplicitBucketBoundariesAdvice() method to set non-default bucket boundaries. Run the application with the java agent, but without the SDK incubator extension.

Full example application is available here: https://github.com/knbk/otel-java-example-project

Expected behavior

The explicit bucket boundaries from the advice are used. In the example app, running gradle app:run should log the test metric with the explicitly configured bucket boundaries.

Actual behavior

The explicit bucket boundaries are discarded by the default method implementation in the interface. The opentelemetry-api-1.31 instrumentation fails to load when the incubator extension is not present:

[otel.javaagent 2023-12-06 10:44:59:415 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - Instrumentation skipped, mismatched references were found: opentelemetry-api [class io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.OpenTelemetryApiInstrumentationModule] on com.blueconic.services.observability [58]
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleGaugeBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleGaugeBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongCounterBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongGaugeBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongGaugeBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongHistogramBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongHistogramBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleCounterBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleHistogramBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleHistogramBuilder
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongGaugeBuilder131$1:0 Missing class io.opentelemetry.extension.incubator.metrics.LongGauge
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleGaugeBuilder131$1:0 Missing class io.opentelemetry.extension.incubator.metrics.DoubleGauge
[otel.javaagent 2023-12-06 10:44:59:428 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleUpDownCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleUpDownCounterBuilder
[otel.javaagent 2023-12-06 10:44:59:429 +0000] [FelixStartLevel] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongUpDownCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongUpDownCounterBuilder

Javaagent or library instrumentation version

v1.32.0

Environment

JDK: Corretto 1.8
OS: Amazon Linux 2

Additional context

No response

[laurit]

@knbk you are only running into this because you are using osgi, right?

[knbk]

@laurit No, this also happens without OSGi, just using the default class loader setup. It only works if I add io.opentelemetry:opentelemetry-sdk-extension-incubator:1.32.0-alpha as a runtime dependency, so that the Extended* classes are available on the application classpath, and not just shaded in the agent class loader. Without the incubator extension, I still get the following messages:

[otel.javaagent 2023-12-06 16:21:09:783 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleGaugeBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleGaugeBuilder
[otel.javaagent 2023-12-06 16:21:09:783 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongCounterBuilder
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongGaugeBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongGaugeBuilder
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongHistogramBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongHistogramBuilder
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleCounterBuilder
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleHistogramBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleHistogramBuilder
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongGaugeBuilder131$1:0 Missing class io.opentelemetry.extension.incubator.metrics.LongGauge
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleGaugeBuilder131$1:0 Missing class io.opentelemetry.extension.incubator.metrics.DoubleGauge
[otel.javaagent 2023-12-06 16:21:09:784 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationDoubleUpDownCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedDoubleUpDownCounterBuilder
[otel.javaagent 2023-12-06 16:21:09:785 +0100] [main] WARN io.opentelemetry.javaagent.tooling.instrumentation.MuzzleMatcher - -- io.opentelemetry.javaagent.instrumentation.opentelemetryapi.v1_31.metrics.ApplicationLongUpDownCounterBuilder131:0 Missing class io.opentelemetry.extension.incubator.metrics.ExtendedLongUpDownCounterBuilder

Let me see if I can push a simple reproducible example.

[knbk]

I've created a small reproducible example here: https://github.com/knbk/otel-java-example-project

Running gradle app:run will not use the bucket boundary advice unless the incubator extension is added to the application.