Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

approval-voting: Add missing traces #6044

Closed
eskimor opened this issue Sep 23, 2022 · 2 comments · Fixed by #6458
Closed

approval-voting: Add missing traces #6044

eskimor opened this issue Sep 23, 2022 · 2 comments · Fixed by #6458
Assignees
Labels
T4-parachains_engineering This PR/Issue is related to Parachains performance, stability, maintenance.

Comments

@eskimor
Copy link
Member

eskimor commented Sep 23, 2022

Grafana Tempo might actually be working, what seems to be missing is traces in various subsystems. It would be good to add those soon, so we can have proper traces in the next debugging session.

@eskimor eskimor added the T4-parachains_engineering This PR/Issue is related to Parachains performance, stability, maintenance. label Sep 23, 2022
@sandreim
Copy link
Contributor

A good start would be approval-voting.

We have spans for handling activated leaves approval-checking-import, approved-ancestor calls, process-approval-wakeup, and launch-approval but we don't get too see all of them in a single view because we are using different identifiers (relay hash, candidate hash, expected_tick, or combinations of these). It is cumbersome at least from Grafna Tempo to have different queries to look at things.

To get a more consolidated view we need to implement the following (non-exhaustive):

  • move spans into per-leaf child spans in to see how much work we are doing per relay block. We would still need to do a separate query if we want to look a specific candidate.
  • adding a child span for assignment imports and nest each import spans under these to observe delay tranches
  • make launch-approval a child span and ensure it fully covers the operation (currently misses 2 other subsystem calls)

@bredamatt
Copy link
Contributor

bredamatt commented Dec 20, 2022

After quite extensively reviewing the tracing tooling we currently use, including whether we could use an alternative library from mick-jaeger such as opentelemetry for example (as a larger task related to revamping how tracing is done), I am currently checking why the ActivatedLeaf (containing the PerLeafSpan) is currently not passed to the approval-voting subsystem. Highlighting that should indicate why the above-mentioned spans are not included in the root leaf-activated span, and subsequently provide the direction to solve this issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
T4-parachains_engineering This PR/Issue is related to Parachains performance, stability, maintenance.
3 participants