[APM] Reparenting spans to support inferred spans

[cauemarcondes]

closes #63575
closes #62952

Reparents spans based on the child_ids property.
Before:

Loader#executeQueryStatement is an Inferred Span which should be child of OpenTracing product span

After:

Adding the child_ids property on OpenTracing product span and reference it Loader#executeQueryStatement, makes the Waterfall to reparent the Loader#executeQueryStatement.

A question that maybe @graphaelli or @felixbarny might know:

• Should I increase the duration of the new parent?
  e.g.:

--[new parent     ] 
----[reparented child     ]  

Should we change it to?

--[new parent             ]
----[reparented child     ]  

[felixbarny]

Is the before screenshot taken from an actual transaction? Or is that a modified one to showcase the representing? I'm asking because I couldn't explain how that situation could have happened, maybe there's a lost activation event for the OpenTracing product span.

Here's another example for what's supposed to happen:

The SELECT FROM customers span is a child of the transaction. But it's supposed to be a child of the MutableEntityEntryFactory#createEntityEntry inferred span.
In this example, the order happens to not be affected by the incorrect parent/child relationships.

I have set profiling_inferred_spans_sampling_interval=5ms for this.

A question that maybe @graphaelli or @felixbarny might know:

Should I increase the duration of the new parent?

TL;DR: No, as normally the Java agent would take care of that.

I'm confused as to why the APIRestController#product span (the 3rd from above) doesn't exceed the OpenTracing product span. I think that is because the activation event of the latter got lost. Otherwise, the Java agent would have corrected the duration of the APIRestController#product span and the Loader#executeQueryStatement span would be a direct child of the OpenTracing product span.

In an ideal world, the two DB spans are supposed to be re-parented to the Loader#executeQueryStatement span. It seems the duration of that span has been auto-aligned by the Java agent to not start after the empty query span and to not end before the SELECT FROM products span.

[cauemarcondes]

Is the before screenshot taken from an actual transaction? Or is that a modified one to showcase the representing?

I got it running apm-integration-test, but since it's not clear for me who should be parent of who I picked any Inferred span, in that case, Loader#executeQueryStatement, and said that it's child of OpenTracing product span.

I also did this because, in my opinion, the UI doesn't have to know about Inferred Spans, it knows how to reparent items based on the child_ids property.

I have set profiling_inferred_spans_sampling_interval=5ms for this.

I'm going to set this variable as 5ms on apm-integration-test and try to find a better example.

[felixbarny]

I also did this because, in my opinion, the UI doesn't have to know about Inferred Spans, it knows how to reparent items based on the child_ids property.

Agreed. For an initial test, this is fine. But a soon as the APM Server supports representing, we should not forget to do another end-to-end test.

[cauemarcondes]

I also did this because, in my opinion, the UI doesn't have to know about Inferred Spans, it knows how to reparent items based on the child_ids property.

Agreed. For an initial test, this is fine. But a soon as the APM Server supports representing, we should not forget to do another end-to-end test.

Agreed. Can you link the APM agent issue here, please?

[felixbarny]

It's linked from the meta issue elastic/apm#247

[dgieselaar]

LGTM!

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 3467807

History

• 💚 Build #41490 succeeded 3467807
• 💚 Build #41222 succeeded 6731e17

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream