setTag not working with OpenTracingShim

[mattspataro]

We're in the process of migrating our codebase from OpenTracing to OpenTelemetry. We're planning on using the OpenTracingShim to ease the transition. However, we're having issues with setting tags during manual instrumentation. Our legacy OpenTracing code will frequently build a span like this:

 tracer
    .buildSpan(Routes.endpointName(request).getOrElse(s"HTTP ${request.method}"))
    .asChildOf(tracer.extract(Format.Builtin.HTTP_HEADERS, new HeadersTextMap(request.headers)))
    .withTag(Tags.SPAN_KIND.getKey, Tags.SPAN_KIND_SERVER)
    .start()

The tags that are added with withTag seem to be added correctly. However, if we try to add tags after a span has been started, then the tags don't seem to stick:
span.setTag("http.response.contentLength", httpResponseContentLength)

It appears like the span that is returned is a SpanShim, and the span within the SpanShim is a io.opentelemetry.api.trace.PropagatedSpan. The PropagatedSpan defined noops for nearly every Span operation (including setTag), which obviously means that no tags are being added. Is there a way to avoid this? What do I need to do so that I can set spans after they have been started?

Any help would be appreciated! Thank you!

[jkwatson]

Can you provide some details about how you're configuring the SDK?

[mattspataro]

This is how we're configuring the SDK:

val attributes = Attributes.builder().put(ResourceAttributes.SERVICE_NAME, hostName.name).build()
val resource = Resource.getDefault.merge(Resource.create(attributes))

val tracerExporter = if (shouldReportApm) OtlpGrpcSpanExporter.builder().build() else NoopSpanExporter
val spanProcessor = BatchSpanProcessor.builder(tracerExporter).build()

val sdkTracerProvider = SdkTracerProvider
  .builder()
  .addSpanProcessor(spanProcessor)
  .setResource(resource)
  .build()

I believe I have stumbled upon a solution, but I'm not sure if it's the best solution.
When I add this .setSampler(Sampler.alwaysOn()) to the SdkTracerProvider, the tags seem to work. For some reason the default ParentBasedSampler determines that child spans shouldn't be sampled, and so the child spans are instantiated as PropagatedSpans rather than real SDK spans. Changing to a AlwaysOnSampler seems to mitigate this issue.

Any idea why this might be happening? Is there a better way to do this so that I can keep the ParentBasedSampler but the parent spans can communicate that they always want to be sampled? Where do the parents determine if they want to be sampled? Any further context that can be given on this would be great!

[jkwatson]

This seems very strange. Can you put together a reproduction case that we could take a look at? It would be helpful to have a fully runnable example that demonstrates the behavior you're seeing, rather than trying to guess at how the whole things is wired up.