Timeline Tracing Roadmap

[zhongzc]

Related Feature Request

#552, pingcap/tidb#17693

Library Development

Many existing tracing libraries are powerful, easy-to-use, and convenient to integrate with other existing components. Unfortunately, almost none of them perform well in terms of performance. For that, lightweight, efficient, and performant tracing libraries for both TiKV and TiDB need to be developed to meet performance requirements. Hopefully, runtime information will be collected constantly so that details of uncommon problems can be caught.

• minitrace-rust for TiKV
  • implement prototype
  • micro and integration benchmark
  • desgin docs (in Chinese)
  • measure time with high performance: lib/minstant
  • support properies: Support property tikv/minitrace-rust#17
  • one-dimensionaize spans to simplify usage: wait time as a span tikv/minitrace-rust#19
• minitrace-go for TiDB
  • implement prototype
  • micro and integration benchmark
    • investigate cost of time measure with different clock sources
  • design docs

Apply to Codebase

After finishing library development, applying the library to the codebase is the next work to do.

• Apply to TiKV
  • collect tracing demands from TiKV: docs (in Chinese)
  • put tracing points to procedures wanted to trace (WIP)
    • handle old async codes of producer/consumer model (WIP)
• Apply to TiDB

Full Integration

Without intervention, all tracing results only reside in memory. However, tracing results of requests associated with a SQL statement or a transaction always need some ways to collect when we intend to show them. These following questions may be of interest to us:

1. How, when, and where to store them?
2. How to fetch them?

Due to the size of tracing results is not small commonly, I/O is always a difficult problem we have to face. It will have a cost that cannot be ignored whether results are persisted to disks or transmitted through the network. Therefore, careful design and meticulous tests are most needed here.

• Handle tracing results from TiKV
  • trade-off docs (in Chinese)
  • gather and send back to TiDB
    • embed the serialized results into grpc responses (WIP)
    • post processing
      • drop some spans if the amount of spans is too big
      • reply or not according to threshold of execution time
    • serialize results to flatbuffers: minitrace-fbs
• Handle all tracing results at TiDB
  • research and benchmark

User Interface

In order to maximize the usability of tracing, a user-friendly UI is essential. People can visually see what's happening during the execution of a SQL statement / a transaction intuitively.

• UX/UI design
• Frontend implementation
• Data fetching and integration