Skip to content

OTLP Log Export, Shared OTLP Connection, Telemetry Accessors

Choose a tag to compare

View all tags
@bw19 bw19 released this 14 Jun 04:46
· 2 commits to main since this release
v1.38.0
db5f64d

v1.38.0 completes Microbus's OpenTelemetry story: logs join traces and metrics as a first-class OTLP signal. When a logs endpoint is configured, every record emitted through the connector's structured logger is exported over OTLP — trace-correlated automatically via native TraceId/SpanId — alongside the existing console output and inline span-event view. Under the hood, all OTLP exporters in one executable that target the same collector now share a single reference-counted connection instead of each opening its own, collapsing up to 3N connections (N services × three signals) down to one per distinct target. The connector also exposes its underlying telemetry handles — Logger(), TracerProvider() and MeterProvider() — so application code can instrument third-party libraries against the same pipelines and resource as the framework. The release is purely additive; there are no breaking changes.

Highlights

  • Logs export over OTLP. When OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (or the generic OTEL_EXPORTER_OTLP_ENDPOINT) is set, the connector exports structured logs to OpenTelemetry. The slog handler fans each record out to the deployment's terminal handler (colorful in LOCAL, text in TESTING, JSON in LAB/PROD) and an otelslog bridge backed by an SDK LoggerProvider, built over the shared OTLP connection with the common resource.
  • Automatic trace correlation for logs. The OTLP bridge stamps native TraceId/SpanId from the context, so backends correlate logs with traces without a string field. The trace string attribute is now added only on the console leg, where it remains the sole correlation handle. Inline span-event mirroring is retained as a separate, in-trace view that survives the PROD tail sampler even with no logs pipeline configured.
  • Shared, reference-counted OTLP connection. All OTLP exporters in one executable that export to the same endpoint over the same protocol share one connection (a gRPC *grpc.ClientConn or a pooled *http.Client), created lazily on first acquire and torn down on last release. A bundle of microservices exporting three signals each would otherwise open up to 3N connections to the collector; sharing collapses that to one per distinct target.
  • Connector.Logger() accessor. Returns the microservice's structured *slog.Logger. Logging through its context-aware methods (InfoContext, ErrorContext, …) enriches each record identically to LogInfo/LogError, because both flow through the same root handler. Before Startup it returns a discard logger, so the accessor never returns nil.
  • TracerProvider() and MeterProvider() accessors. Return the live SDK providers — or a no-op provider (never nil) when the signal is disabled — so application code can instrument OTEL-aware third-party libraries against the framework's pipelines and resource.
  • Per-call log enrichment moved into the handler. Trace-ID and actor-claim stamping, span-event mirroring, the microbus_log_messages count, and developer-deployment stderr error dumps now live in the connector's root slog handler rather than in each LogXXX method body. microbus_log_messages consequently counts every record emitted through the logger, including a bare Logger().Info(...), not just the convenience methods.
  • OTLP transport security from the endpoint and env. Transport security follows the endpoint URL scheme (https is secured), with OTEL_EXPORTER_OTLP[_signal]_INSECURE overriding it. Because the connector now builds the connection itself, it also honors the OTLP certificate env vars directly: a custom CA (..._CERTIFICATE) and a client certificate for mTLS (..._CLIENT_CERTIFICATE / ..._CLIENT_KEY).
  • Ordered shutdown teardown. The final "Shutdown" log is recorded while the meter and tracer are still live (it feeds both a log counter and a span event), then the providers are torn down in reverse of init order — logs, then traces, then metrics — so that last log's counter increment and span event are captured rather than dropped.
  • gRPC is now a direct dependency, attributed in ATTRIBUTION.md, since the connector dials the shared OTLP gRPC connection itself.

New Features

OTLP Log Export

Set a logs endpoint — per-signal or via the generic endpoint that drives all three signals — to stream structured logs to an OpenTelemetry collector:

# env.yaml
# The generic endpoint drives all three signals (traces, metrics, logs).
OTEL_EXPORTER_OTLP_ENDPOINT: http://127.0.0.1:4317
# ...or set just the logs signal to override one:
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: http://127.0.0.1:4317

No code change is required. Logging continues through the same LogInfo/LogWarn/LogError/LogDebug methods (or Logger()), and each record is now mirrored to the collector in addition to the console and the active span. The otelslog bridge stamps the trace and span IDs natively, so logs land already correlated with their traces in Grafana's LGTM stack. Export is best-effort sideband: a configured-but-unreachable collector never affects service health — exports are single-shot (retries disabled) and time out per OTEL_EXPORTER_OTLP_TIMEOUT rather than stalling startup or shutdown.

Telemetry Provider Accessors

Three accessors expose the connector's underlying telemetry handles so application code can wire third-party, OTEL-instrumented libraries into the same pipelines and resource the framework uses:

// Instrument a third-party library against the framework's own pipelines.
logger := svc.Logger()                  // *slog.Logger, enriched like LogXXX via *Context methods
tracer := svc.TracerProvider().Tracer("my-lib")
meter  := svc.MeterProvider().Meter("my-lib")

Logger() returns a discard logger before Startup; TracerProvider() / MeterProvider() return a no-op provider when their signal is disabled. None of the three ever returns nil. Fetch them at point of use rather than caching across Startup — the placeholders are replaced with the real handles during Startup. They are concrete methods on the connector and are reachable via the embedded connector on the concrete *Service.

Shared OTLP Connection

A registry in the connector keys shared connections on protocol|endpoint|caFile|clientCert|clientKey|insecure. The TLS material and the insecure override are part of the key, so two signals pointed at the same endpoint but secured differently never alias onto one connection. Each signal of each connector holds one reference; the connection is dialed on first export (preserving the non-blocking-Startup guarantee) and closed on last release. The connector — not the SDK — owns the connection: exporters are constructed with WithGRPCConn / WithHTTPClient, and the SDK never closes a caller-provided connection, so one connector's shutdown cannot pull the connection out from under another that still holds a reference.

Breaking Changes

None. v1.38.0 is purely additive — the new accessors and the logs signal layer on top of the existing logging, tracing, and metrics APIs. The slog handler internals were reorganized, but the service.Logger interface (LogDebug/LogInfo/LogWarn/LogError) is unchanged. The only observable behavior change is that microbus_log_messages now counts records emitted via a bare Logger().Info(...) in addition to the convenience methods.

Migration

From inside a Microbus project, ask Claude Code to upgrade Microbus:

{{< prompt >}}
Get the latest version of Microbus.
{{< /prompt >}}

The upgrade skill handles the version bump end-to-end:

  1. Bump go.mod to v1.38.0 and go mod tidy.
  2. Refresh .claude/rules/, .claude/skills/, and project-wide framework-managed files.
  3. Run go vet ./... && go test ./....

No source rewrites are required. To start exporting logs over OTLP, add OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (or rely on the generic OTEL_EXPORTER_OTLP_ENDPOINT) to env.yaml.

Documentation

  • Updated: Structured Logging for OTLP log export, automatic trace correlation, and the Logger() accessor.
  • Updated: Distributed Tracing and Metrics for the TracerProvider() / MeterProvider() accessors and the shared OTLP connection.
  • Updated: Environment Variables for the logs endpoint and the OTLP transport-security env vars (..._INSECURE, ..._CERTIFICATE, ..._CLIENT_CERTIFICATE / ..._CLIENT_KEY).
  • Updated: Package connector for the new otel.go shared-connection helpers.
Assets 2
Loading