Support for using tracing span fields as metrics labels #87

MOZGIII · 2020-09-04T06:43:27Z

This PR adds support for using values defined via the tracing::span! macro as labels in the metrics crate.

It builds upon the refactor/metrics-v2 branch (metrics 0.13 series).

Closes #70
Closes #89

Current status

discussion is ongoing at https://discord.gg/RqTk6bq;
PoC with a workaround for callsite optimization is implemented;
API switched to use Key (a container KeyData) instead of Identifier to allow proper registry swap and enable decoupling required for altering the Key at runtime (at Layers);
Registry has been reimplemented to better fit the new Recorder API;
simple and extensible label filtering is implemented;
~~review in progress~~ merged.

To do

get approval on the design
uncover edge cases
improve the UX
- dynamic label filtering
- ~~explicit definitions of the allowed dynamic labels list at the macros~~
optimize Registry API and impl for the new operation mode
add proper test coverage
add proper docs
bench the macros
bench the Registry

It doesn't use Registry at all anymore!

The iteration, unfortunately, can only happen in the inside-out order of span nesting. We will work around this on another level, and here we just collect all the labels out there.

tobz

Mostly just a lot of nits about documentation formatting/spelling.

metrics-macros/src/lib.rs

metrics-tracing-context/src/label_filter.rs

metrics-tracing-context/src/lib.rs

metrics-tracing-context/src/tracing_integration.rs

metrics/src/key.rs

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

MOZGIII added 5 commits September 4, 2020 09:38

Add editorconfig

2ee2633

Add initial metrics-tracing-context version

b787867

Add test_no_labels

5494cf2

Add test_multiple_paths_to_the_same_callsite (fails currently)

dbc257e

Make integration tests runnable in bulk

f75e0e5

tobz mentioned this pull request Sep 6, 2020

Support for using tracing span fields as metrics labels (dynamic labels remix) #88

Closed

MOZGIII added 6 commits September 9, 2020 06:13

Add reverse mappings to registry

8794c05

Handle dynamic labels explicitly in the interface and macros

5b613b4

Provide a default implementation for dynamic metric ops

529f3fe

Update metrics-util

3038fd8

Update metrics-tracing-context

0f1fb9c

Apply fmt

809d626

MOZGIII mentioned this pull request Sep 9, 2020

Include internal trace context with internal metrics vectordotdev/vector#2660

Closed

MOZGIII added 13 commits September 10, 2020 11:13

Fix handling of labels during registration

eab67dd

Add tests for macros

4021182

Move identifier into a separate mod

951b1a9

Correct the test at metrics-tracing-context/tests/integration.rs

2778367

Switch to using Key instead of Identifier for usual calls

aae4036

Update utils to use Key instead of Identifier

0114065

fixup! Switch to using Key instead of Identifier for usual calls

3782b9d

Update metrics-tracing-context to make the tests pass

6dd5643

Update the exporter crates to use Key instead of Identifier

3021b49

Apply a workaround for unstable ordering of the snapshot output

dc5652d

Update the bench code with the latest design

516a876

Apply workaround for criterion comparison

48d6af8

Completely switch to KeyRef at the API and re-add callsite caching

d7c7523

MOZGIII force-pushed the refactor/metrics-v2-tracing-context branch from 2d7081c to d7c7523 Compare September 11, 2020 07:12

MOZGIII added 3 commits September 11, 2020 10:27

Removed the option wrapper at Key::labels internally

14ca038

Removed the option wrapper at Key::labels in the public API

cbdb4e9

More performant can_use_fast_path

ea06aba

MOZGIII added 9 commits September 14, 2020 14:23

Better notation for owned key at registry benches

77f1b3a

Add criterion workaround at metrics-util/Cargo.toml

6cc1339

Change the Registry for upsert-like API and use DashMap internally

57698c2

Update prometheus to the new Registry API

261474e

Update the metrics-exporter-tcp for new API

d2d6743

It doesn't use Registry at all anymore!

Update registry benches

71b4b71

Change the test_nested_spans

4142a54

The iteration, unfortunately, can only happen in the inside-out order of span nesting. We will work around this on another level, and here we just collect all the labels out there.

Implement the code to make the test_nested_spans pass

fa727ef

Add label filtering

ea2e319

MOZGIII marked this pull request as ready for review September 14, 2020 20:06

More registry benches

ed1c0d9

tobz requested changes Sep 15, 2020

View reviewed changes

MOZGIII and others added 6 commits September 15, 2020 14:37

Update metrics/src/key.rs

cfd00ef

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

Update metrics/src/key.rs

b877f9b

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

Update metrics-tracing-context/src/lib.rs

eb08c4a

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

Update metrics-tracing-context/src/label_filter.rs

406f515

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

Update metrics-macros/src/lib.rs

c32b5c3

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

Update metrics-macros/src/lib.rs

dda51e1

Co-authored-by: Toby Lawrence <tobz@users.noreply.github.com>

MOZGIII requested a review from tobz September 15, 2020 11:40

Add proper trailing comma handling and parsing tests

339dc25

MOZGIII mentioned this pull request Sep 15, 2020

A way to force a particular a span! to be always enabled regardless of the rest of the configuration tokio-rs/tracing#978

Open

Better crate doctest

79060bc

MOZGIII force-pushed the refactor/metrics-v2-tracing-context branch from 3f5626e to 79060bc Compare September 15, 2020 22:53

tobz mentioned this pull request Sep 16, 2020

Optimize Visit implementation for tracing context #90

Closed

tobz approved these changes Sep 16, 2020

View reviewed changes

tobz merged commit 8bbe1b1 into metrics-rs:refactor/metrics-v2 Sep 16, 2020

MOZGIII mentioned this pull request Sep 16, 2020

chore: Use span fields as metrics labels vectordotdev/vector#3888

Merged

This was referenced Sep 16, 2020

Metrics redesign: labels via context #70

Closed

Key construction from &str always allocate #50

Closed

Key construction from labels always allocate #49

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for using tracing span fields as metrics labels #87

Support for using tracing span fields as metrics labels #87

MOZGIII commented Sep 4, 2020 •

edited

Loading

tobz left a comment

Support for using tracing span fields as metrics labels #87

Support for using tracing span fields as metrics labels #87

Conversation

MOZGIII commented Sep 4, 2020 • edited Loading

Current status

To do

tobz left a comment

Choose a reason for hiding this comment

MOZGIII commented Sep 4, 2020 •

edited

Loading