Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentry Spring WebFlux integration creates a new hub and scope which is not used for OpenTelemetry transactions #2726

Open
adinauer opened this issue May 23, 2023 Discussed in #2715 · 0 comments

Comments

@adinauer
Copy link
Member

Discussed in #2715

Problem

SentrySpanProcessor creates a new transaction for a request which is bound to the current hub.
SentryWebFilter then creates a new hub cloned from the main hub. This causes a new and separate scope to be created. If a developer now manipulates the scope during the request, these changes are lost.

Possible Fixes

Creating a new hub in SentrySpanProcessor

  • Will probably reintroduce OOM issues we had before as we can't ensure we'll find the same hub again during request handling which may lead to an ever growing stack of scopes eventually causing OOM.

Could store the hub in SentrySpanStorage and then look it up using OpenTelemetry span id

  • Would probably require some generic mechanism in options that could be set by our OpenTelemetry integration where instead of creating a new hub and pushing a new scope we look up the hub in SentrySpanStorage.
  • If not using a replacable mechanism we'd be tying the WebFlux integration to OpenTelemetry or would have to offer yet another package which is suboptimal

Workarounds

Won't affect Sentry integrations. Please give those a try and give us feedback.

Manually manipulation the Sentry transaction directly

import io.opentelemetry.api.trace.Span;
import io.sentry.ISpan;
import io.sentry.ITransaction;
import io.sentry.Sentry;
import io.sentry.SentrySpanStorage;
import io.sentry.SpanId;

  @GetMapping("{id}")
  Person person(@PathVariable Long id) {
    ISpan span = Sentry.getCurrentHub().getSpan().startChild("op1");
    try {
      @Nullable ITransaction transaction = TransactionFinder.findTransaction();
      transaction.setData("my-data-key", "my-data-value");
      System.out.println(transaction);
      LOGGER.info("Loading person with id={}", id);
      throw new IllegalArgumentException("Something went wrong [id=" + id + "]");
    } finally {
      span.finish();
    }
  }

  static class TransactionFinder {
    public static @Nullable ITransaction findTransaction() {
      String otelSpanId = Span.current().getSpanContext().getSpanId();
      return findTransaction(otelSpanId);
    }

    private static @Nullable ITransaction findTransaction(@Nullable String spanId) {
      if (spanId == null) {
        return null;
      }

      @Nullable ISpan span = SentrySpanStorage.getInstance().get(spanId);

      if (span instanceof ITransaction) {
        return (ITransaction) span;
      }

      if (span instanceof io.sentry.Span) {
        SpanId parentSpanId = ((io.sentry.Span) span).getParentSpanId();
        if (parentSpanId != null) {
          return findTransaction(parentSpanId.toString());
        }
      }

      return null;
    }
  }

Manually manipulating the OpenTelemetry span

Span.current().setAttribute("my-otel-attr", "otel-attr-value");

For manipulating things like request etc. this could also be done via EventProcessor or beforeSend / beforeSendTransaction:

  @Bean
  EventProcessor eventProcessor() {
    return new EventProcessor() {
      @Override
      public @Nullable SentryEvent process(@NotNull SentryEvent event, @NotNull Hint hint) {
        addOtelAttr(event, hint);
        return event;
      }

      @Override
      public @Nullable SentryTransaction process(@NotNull SentryTransaction transaction, @NotNull Hint hint) {
        addOtelAttr(transaction, hint);
        return transaction;
      }

      @SuppressWarnings("unchecked")
      private void addOtelAttr(final @NotNull SentryBaseEvent event, final @NotNull Hint hint) {
        Object otelContextObject = event.getContexts().get("otel");
        if (otelContextObject instanceof Map) {
          Map<String, Object> otelContext = (Map<String, Object>) otelContextObject;
          Object attributesObject = otelContext.get("attributes");
          if (attributesObject instanceof Map) {
            Map<String, Object> attributes = (Map<String, Object>) attributesObject;
            Object myValue = attributes.get("my-otel-attr");
            event.setExtra("my-extra-from-otel", myValue);
          }
        }
      }
    };
  }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant