Skip to content

[common/tracing] feature: report distributed tracing stat to jaeger#1226

Merged
databend-bot merged 2 commits into
databendlabs:masterfrom
drmingdrmer:jaeger-tracing
Jul 29, 2021
Merged

[common/tracing] feature: report distributed tracing stat to jaeger#1226
databend-bot merged 2 commits into
databendlabs:masterfrom
drmingdrmer:jaeger-tracing

Conversation

@drmingdrmer

@drmingdrmer drmingdrmer commented Jul 29, 2021

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://datafuse.rs/policies/cla/

Summary

[common/tracing] feature: report distributed tracing stat to jaeger

Start a local jaeger server and report tracing data and view it:

docker run -d -p6831:6831/udp -p6832:6832/udp -p16686:16686 jaegertracing/all-in-one:latest
FUSE_JAEGER=on RUST_LOG=trace cargo test
open http://localhost:16686/
  • Add: report tracing data to opentelemetry server(jaeger) with env
    FUSE_JAEGER=on

  • Add tracing::instrument to store flight server and meta server.

  • Inject tracing span to tonic request on the client side
    and extract them on the server side to form a complete tracing.

  • Add tracing to SledTree

截屏2021-07-29 下午1 57 40

TODO:

  • Sometimes the spans messes up with the parent-child relationship. Maybe caused by abnormal thread switching?
  • Chaining the span when a work was sent through channel etc.

Changelog

  • New Feature

  • Improvement

Related Issues

#271

Start a local jaeger server and report tracing data and view it:

```
docker run -d -p6831:6831/udp -p6832:6832/udp -p16686:16686 jaegertracing/all-in-one:latest
FUSE_JAEGER=on RUST_LOG=trace cargo test
open http://localhost:16686/
```

-   Add: report tracing data to opentelemetry server(jaeger) with env
    `FUSE_JAEGER=on`

-   Add tracing::instrument to store flight server and meta server.

-   Inject tracing span to tonic request on the client side
    and extract them on the server side to form a complete tracing.

-   Add tracing to SledTree
@drmingdrmer drmingdrmer added this to the v0.5 milestone Jul 29, 2021
@databend-bot databend-bot added pr-feature this PR introduces a new feature to the codebase pr-improvement labels Jul 29, 2021
@databend-bot

Copy link
Copy Markdown
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

3 similar comments
@databend-bot

Copy link
Copy Markdown
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@databend-bot

Copy link
Copy Markdown
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@databend-bot

Copy link
Copy Markdown
Member

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

@drmingdrmer
drmingdrmer requested a review from bohutang July 29, 2021 06:36
@drmingdrmer drmingdrmer changed the title [common/tracing] feature: report tracing stat to jaeger [common/tracing] feature: report distributed tracing stat to jaeger Jul 29, 2021
@drmingdrmer
drmingdrmer marked this pull request as ready for review July 29, 2021 06:37
@drmingdrmer

Copy link
Copy Markdown
Member Author

Several TODO yet:

  • Sometimes the spans messes up with the parent-child relationship. Maybe caused by abnormal thread switching?
  • Chaining the span when a work was sent through channel etc.
  • Use batch mode to speed up tracing data sending.
  • Use env var FUSE_JAEGER to specify a dedicate jaeger server or provide some way else.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

Merging #1226 (0424aa1) into master (b6d9835) will increase coverage by 0%.
The diff coverage is 95%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #1226    +/-   ##
=======================================
  Coverage      72%     72%            
=======================================
  Files         464     464            
  Lines       27678   27997   +319     
=======================================
+ Hits        20076   20354   +278     
- Misses       7602    7643    +41     
Impacted Files Coverage Δ
fusestore/store/src/meta_service/network.rs 47% <50%> (-2%) ⬇️
...estore/store/src/meta_service/meta_service_impl.rs 56% <75%> (-1%) ⬇️
fusestore/store/src/meta_service/sled_tree.rs 96% <95%> (+4%) ⬆️
fusestore/store/src/api/rpc/flight_service_test.rs 97% <99%> (+1%) ⬆️
common/flights/src/impls/kv_api_impl.rs 79% <100%> (-1%) ⬇️
common/flights/src/store_client.rs 85% <100%> (+2%) ⬆️
fusestore/store/src/api/rpc/flight_service.rs 60% <100%> (+3%) ⬆️
fusestore/store/src/meta_service/raft_log.rs 96% <100%> (+<1%) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6d9835...0424aa1. Read the comment docs.

@bohutang

Copy link
Copy Markdown
Member

Several TODO yet:

  • Sometimes the spans messes up with the parent-child relationship. Maybe caused by abnormal thread switching?
  • Chaining the span when a work was sent through channel etc.
  • Use batch mode to speed up tracing data sending.
  • Use env var FUSE_JAEGER to specify a dedicate jaeger server or provide some way else.

Wow, good-first-issue please

@bohutang bohutang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@databend-bot

Copy link
Copy Markdown
Member

CI Passed
Reviewer Approved
Let's Merge

@databend-bot
databend-bot merged commit 7934cda into databendlabs:master Jul 29, 2021
@drmingdrmer
drmingdrmer deleted the jaeger-tracing branch July 29, 2021 07:14
@drmingdrmer

Copy link
Copy Markdown
Member Author

Several TODO yet:

  • Sometimes the spans messes up with the parent-child relationship. Maybe caused by abnormal thread switching?
  • Chaining the span when a work was sent through channel etc.
  • Use batch mode to speed up tracing data sending.
  • Use env var FUSE_JAEGER to specify a dedicate jaeger server or provide some way else.

Wow, good-first-issue please

Aye

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants