Little bit lost with opentelemetry tracing context

[Dr-Dream]

Friends i'm trying 1.0.0 version with opentelemetry traces enabled.
My workflows triggered by CloudEvent
So when i look in to logs i see pretty expected traceId and spanid

18:43:02 INFO  traceId=12f121452c52dc1ad6f3ccdca1c85196, parentId=, spanId=3309b808d41fcaa4, sampled=true [io.qu.re.ht.ru.HttpCloudEventHelper] (vert.x-eventloop-thread-5) Unrecognized CE attribute ce-chattype, assuming extension
18:43:02 INFO  traceId=12f121452c52dc1ad6f3ccdca1c85196, parentId=, spanId=3309b808d41fcaa4, sampled=true [io.qu.re.ht.ru.HttpCloudEventHelper] (vert.x-eventloop-thread-5) Unrecognized CE attribute ce-chatid, assuming extension
18:43:02 INFO  traceId=12f121452c52dc1ad6f3ccdca1c85196, parentId=, spanId=3309b808d41fcaa4, sampled=true [io.qu.re.ht.ru.HttpCloudEventHelper] (vert.x-eventloop-thread-5) Unrecognized CE attribute ce-messageid, assuming extension

But all other tasks and logs are going without a trace

18:43:02 DEBUG traceId=, parentId=, spanId=, sampled= [io.qu.fl.me.FlowMessagingConsumer] (pool-1-thread-1) Flow: Received event: CloudEvent{id='4def0d1b-1cb0-48c8-93b2-d619fda90a81', source=/my-payment-gateway/test, 
......
18:43:17 DEBUG traceId=, parentId=, spanId=, sampled= [io.qu.fl.op.ru.OTelWorkflowExecutionListener] (executor-thread-25) On - TASK_COMPLETED: taskName: generateImage, taskType: emit, taskId: do/3/generateImage, iteration: 1, isRetrying: false, retryAttempt: 0, retryCount: 0
18:43:17 INFO  traceId=, parentId=, spanId=, sampled= [io.qu.fl.tr.TraceLoggerExecutionListener] (executor-thread-30) Task 'finishTask' started at 2026-08-27T18:43:17.823807+03:00 pos=do/4/finishTask
18:43:17 DEBUG traceId=, parentId=, spanId=, sampled= [io.qu.fl.op.ru.OTelWorkflowExecutionListener] (executor-thread-30) On - TASK_STARTED: taskName: finishTask, taskType: emit, taskId: do/4/finishTask, iteration: 1, isRetrying: false, retryAttempt: 0, retryCount: 0

So i suppose parent context lost somewhere on workflow execution. Suppose it is io.serverlessworkflow.impl.WorkflowMutableInstance#startExecution

But! When i look into tempo i see full trace of workflow.

So i have 2 different traces
First one is for HTTP POST (cloudevent)
And second one for flow itself.

So i have several questions:

1. Did i missed something in propagation? Or it is something that should be fixed? (Suppose issue is somewhere near io.serverlessworkflow.impl.WorkflowMutableInstance#startExecution)
2. How it will work with persistent flow which is resumed after restart? Did tracing context saved in metadata?
3. Thinking of events correlation it is not quite clear for me what should be when flow initiated with one parent trace but correlated events are coming with another trace? Suppose original traceId should be used.
4. Is there any way to get relation between span and flow logs?

[mathecruz]

Hello @Dr-Dream, thanks for opening this discussion, could you share with us a reproducer?

1. The reproducer will help me to answer it in more details and to understand the current behavior.. I would like to know what extensions are you using, your workflow, etc.

2. No, currently the traces lives only in memory, we are not persisting it to the DB.

3. It is not handled today, we are not reading it. I am not sure if doing a re-parent is the better choice, maybe a Link is the better approach.

4. Currently it is not shared today, traces and logs are separated.

Anyway, thanks again for reporting this, we will create an ADR to address the points shared with us.

[mathecruz]

@Dr-Dream about the reproducer I created one based in your logs, the disconnected tracing reported by you will be fixed here, track it until a new version.

[Dr-Dream]

Thanks. I was supposing same that issue comming with http connector.
So now i was trying just to pass cloudevent over http (as knative do) in more generic way.

[ricardozanini]

How it will work with persistent flow which is resumed after restart? Did tracing context saved in metadata?

It's not saved, but all the traces that were already published will remain, and once resume happens, you will see the rest of the trace.

@wmedvede please have a look.

[mathecruz]

Hey @wmedvede, I tried to reproduce the behavior here, take a look.

[Dr-Dream]

I'll check my setup. I did it without connecting directly to physical broker but using generic knative brokers (pure http calls)
I will continue investigation in my lab.

[ricardozanini]

@mcruzdev I understand we have several PRs/issues open to handle this use case. Can you please add them here as a table summarizing each responsibility?

[mathecruz]

• About the discorrelated traces, we already identified where the issue lives and we have a pull request opened to fix it.
  • Issue: Incoming HTTP messages carry no OpenTelemetry trace context (unlike the Kafka connector) quarkus-reactive-messaging-http#442
  • Pull request quarkus-reactive-messaging-http

All other items mentioned here are being handled by the OpenTelemetry Trace Propagation Across CloudEvent-Triggered Workflow Execution - Design Specification.

Once the ADR pull request is merged, I will open three independent issues, one for each problem reported (which can be solved in parallel), below we can see the updated status of items 2-4:

2. How it will work with persistent flow which is resumed after restart? Did tracing context saved in metadata?

• This solution requires a change in the engine (sdk-java) and we are mapping it through the following issue.
  • Cross-cutting data attached to a workflow instance by WorkflowExecutionListener should survive process restarts open-workflow-specification/sdk-java#1645
  • OpenTelemetry: Broken Trace Continuity for Durable Workflows After JVM Restart #1040

3. Thinking of events correlation it is not quite clear for me what should be when flow initiated with one parent trace but correlated events are coming with another trace? Suppose original traceId should be used.

4. Is there any way to get relation between span and flow logs?

• Correlate flow lifecycle logs with the active OpenTelemetry trace #1013

It is being handled into the ADR mentioned above. Once we have the issue I will put here

[wmedvede]

@ricardozanini btw, things like "How it will work with persistent flow which is resumed after restart? Did tracing context saved in metadata?", were never planned for iteration 1, and instead will be part of my work for iteration 2.