Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: More granular instrumentation and remove long running spans #811

Merged
merged 29 commits into from
Jun 21, 2024

Conversation

NickLarsenNZ
Copy link
Member

@NickLarsenNZ NickLarsenNZ commented Jun 12, 2024

Description

Full chain

Note

All spans are in the same trace when the client has not sent a trace ID in the request.

image

image

Linked traces

When a client sends trace information, we can pick that up and use the trace context from there. Notice the traceparent send is the same one as what is received. The webhook is using the clients trace context from thereon.

image

Note

Unfortunately trace ids cannot be updated on spans, so once we are able to extract the trace-id from the headers of a client request, we can only use it from that point forward.
But, we are able to add links to spans to make life easier.

Client makes a request, and injects the trace-id into headers.
A link appears on the span where the parent context was updated.

image

Clicking on the link will take you to the trace with the spans that existed before the trace-id was extracted from the headers.

image

This trace also has a span with a link back to the other.

Similarly in Jaeger

image

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Reviewer

Edit tasklist title
Beta Give feedback Tasklist Reviewer, more options

Delete tasklist

Delete tasklist block?
Are you sure? All relationships in this tasklist will be removed.
  1. Code contains useful comments
    Options
  2. Documentation added or updated
    Options
  3. Changelog updated
    Options
  4. Cargo.toml only contains references to git tags (not specific commits or branches)
    Options
Loading

Defaults are:

```
            sampler: Box::new(Sampler::ParentBased(Box::new(Sampler::AlwaysOn))),
            id_generator: Box::<RandomIdGenerator>::default(),
```
… handling, and high level route handling

This fixes the trace hierarchy which used to be disjoint because the parent span/trace id was missing and a new one was being generated in the axum layers
@NickLarsenNZ NickLarsenNZ self-assigned this Jun 12, 2024
@NickLarsenNZ NickLarsenNZ marked this pull request as ready for review June 12, 2024 17:05
Copy link
Member

@Techassi Techassi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just some questions and nitpicks.

crates/stackable-webhook/src/tls.rs Outdated Show resolved Hide resolved
crates/stackable-webhook/src/tls.rs Outdated Show resolved Hide resolved
crates/stackable-webhook/src/tls.rs Show resolved Hide resolved
crates/stackable-telemetry/src/tracing.rs Show resolved Hide resolved
crates/stackable-telemetry/src/instrumentation/axum/mod.rs Outdated Show resolved Hide resolved
@Techassi Techassi self-requested a review June 21, 2024 11:45
Copy link
Member

@Techassi Techassi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just some small remarks.

crates/stackable-operator/CHANGELOG.md Outdated Show resolved Hide resolved
crates/stackable-telemetry/CHANGELOG.md Outdated Show resolved Hide resolved
crates/stackable-telemetry/src/instrumentation/axum/mod.rs Outdated Show resolved Hide resolved
crates/stackable-telemetry/src/tracing.rs Show resolved Hide resolved
@NickLarsenNZ NickLarsenNZ added this pull request to the merge queue Jun 21, 2024
Merged via the queue into main with commit 7bc8489 Jun 21, 2024
17 checks passed
@NickLarsenNZ NickLarsenNZ deleted the remove-long-spans branch June 21, 2024 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants