time to establish connection #3134

John-LittleBearLabs · 2022-11-17T17:40:19Z

Description

Implementing #2745 , adding a metric to break down time from connection pending to connection established, per protocol stack.

$curl -s http://127.0.0.1:42183/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Listener",protocols="/ip4/tcp"} 0.007
libp2p_swarm_connection_establishment_duration_count{role="Listener",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.008"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="+Inf"} 1

lbl@chomp:~lbl
$curl -s http://127.0.0.1:34283/metrics | grep nt_duration
# HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections.
# TYPE libp2p_swarm_connection_establishment_duration histogram
libp2p_swarm_connection_establishment_duration_sum{role="Dialer",protocols="/ip4/tcp"} 0.009
libp2p_swarm_connection_establishment_duration_count{role="Dialer",protocols="/ip4/tcp"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.001"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.002"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.004"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.008"} 0
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.016"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.032"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.064"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.128"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.256"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.512"} 1
libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="+Inf"} 1

Notes

Buckets subject to tweak.

Links to any relevant issues

#2745

Open Questions

The Histogram being used here appears to be a cumulative histogram. Is that good enough?

I suspect that it is, since the purer measurements are deducible, but it's mildly disappointing.

Change checklist

I have performed a self-review of my own code
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
A changelog entry has been made in the appropriate crates

mxinden

A couple of comments. Direction looks good to me.

misc/metrics/src/swarm.rs

protocols/autonat/tests/test_server.rs

swarm/src/connection/pool.rs

Co-authored-by: Max Inden <mail@max-inden.de>

…bs/rust-libp2p into jt/2745-time2establish

clippy.toml

misc/metrics/src/swarm.rs

As per suggestion. Co-authored-by: Max Inden <mail@max-inden.de>

mxinden

Overall this looks good to me. Just some smaller discussions.

clippy.toml

misc/metrics/src/swarm.rs

swarm/src/connection/pool.rs

Co-authored-by: Max Inden <mail@max-inden.de>

mxinden

Just small nits. Otherwise this looks good to me.

misc/metrics/src/swarm.rs

swarm/src/lib.rs

swarm/src/connection/pool.rs

mxinden

Can you also please add a changelog entry like you did in #2982?

I will take care of the changelog entry and version bumps related to the breaking change in libp2p-swarm.

misc/metrics/src/swarm.rs

swarm/src/connection/pool.rs

checkpoint

mergify · 2022-12-12T11:32:33Z

This pull request has merge conflicts. Could you please resolve them @John-LittleBearLabs? 🙏

mxinden

Thanks @John-LittleBearLabs!

Implementing libp2p#2745 , adding a metric to break down time from connection pending to connection established, per protocol stack. ```` $curl -s http://127.0.0.1:42183/metrics | grep nt_duration # HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections. # TYPE libp2p_swarm_connection_establishment_duration histogram libp2p_swarm_connection_establishment_duration_sum{role="Listener",protocols="/ip4/tcp"} 0.007 libp2p_swarm_connection_establishment_duration_count{role="Listener",protocols="/ip4/tcp"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.001"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.002"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.004"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.008"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.016"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.032"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.064"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.128"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.256"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="0.512"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Listener",protocols="/ip4/tcp",le="+Inf"} 1 lbl@chomp:~lbl $curl -s http://127.0.0.1:34283/metrics | grep nt_duration # HELP libp2p_swarm_connection_establishment_duration Time it took (locally) to finish establishing connections. # TYPE libp2p_swarm_connection_establishment_duration histogram libp2p_swarm_connection_establishment_duration_sum{role="Dialer",protocols="/ip4/tcp"} 0.009 libp2p_swarm_connection_establishment_duration_count{role="Dialer",protocols="/ip4/tcp"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.001"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.002"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.004"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.008"} 0 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.016"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.032"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.064"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.128"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.256"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="0.512"} 1 libp2p_swarm_connection_establishment_duration_bucket{role="Dialer",protocols="/ip4/tcp",le="+Inf"} 1 ````

Buckets now exponential, by request.

bac7601

mxinden reviewed Nov 17, 2022

View reviewed changes

John-LittleBearLabs and others added 8 commits November 17, 2022 13:04

Update misc/metrics/src/swarm.rs

221a9ba

Co-authored-by: Max Inden <mail@max-inden.de>

Update swarm/src/connection/pool.rs

582ca19

Co-authored-by: Max Inden <mail@max-inden.de>

Update swarm/src/connection/pool.rs

4e330c5

Co-authored-by: Max Inden <mail@max-inden.de>

Update swarm/src/connection/pool.rs

7d1f76c

Co-authored-by: Max Inden <mail@max-inden.de>

Absorbing recommendations.

a00eb9e

Merge branch 'jt/2745-time2establish' of github.com:John-LittleBearLa…

5825d3f

…bs/rust-libp2p into jt/2745-time2establish

Taking some suggestions.

0deb409

clippy

44ef2fd

mxinden reviewed Nov 18, 2022

View reviewed changes

clippy.toml Outdated Show resolved Hide resolved

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

Update misc/metrics/src/swarm.rs

a9dad92

As per suggestion. Co-authored-by: Max Inden <mail@max-inden.de>

mxinden reviewed Nov 23, 2022

View reviewed changes

clippy.toml Outdated Show resolved Hide resolved

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

swarm/src/connection/pool.rs Outdated Show resolved Hide resolved

swarm/src/connection/pool.rs Outdated Show resolved Hide resolved

John-LittleBearLabs and others added 4 commits November 23, 2022 09:19

Update swarm/src/connection/pool.rs

4e2cc29

Co-authored-by: Max Inden <mail@max-inden.de>

Remove clippy config, as it is not pertinent.

2be9db9

Forgot to push this rename

60a0429

Merge remote-tracking branch 'origin/master' into jt/2745-time2establish

a6d375d

mxinden reviewed Dec 5, 2022

View reviewed changes

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

swarm/src/lib.rs Outdated Show resolved Hide resolved

mxinden marked this pull request as ready for review December 5, 2022 13:00

Naming suggestions.

a1a4fbf

mxinden reviewed Dec 6, 2022

View reviewed changes

swarm/src/lib.rs Outdated Show resolved Hide resolved

swarm/src/connection/pool.rs Outdated Show resolved Hide resolved

naming

a9d26e5

mxinden reviewed Dec 8, 2022

View reviewed changes

misc/metrics/src/swarm.rs Outdated Show resolved Hide resolved

swarm/src/connection/pool.rs Outdated Show resolved Hide resolved

John-LittleBearLabs added 3 commits December 8, 2022 09:26

elapsed()

88ae127

Changelog

bd0874c

squash! Changelog

e4b582b

checkpoint

mxinden added 2 commits December 12, 2022 12:57

Merge branch 'master' into jt/2745-time2establish

655dec4

swarm/CHANGELOG: Add entry

5a68ae1

mxinden approved these changes Dec 12, 2022

View reviewed changes

mxinden added the send-it label Dec 12, 2022

Merge remote-tracking branch 'origin/master' into jt/2745-time2establish

e60d9de

mergify bot merged commit f8f19ba into libp2p:master Dec 12, 2022

mxinden mentioned this pull request Dec 16, 2022

fix(metrics): Update connections_establishment_duration buckets #3256

Merged

4 tasks

thomaseizinger mentioned this pull request Jun 15, 2023

swarm/: Expose time to establish connection #2745

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

time to establish connection #3134

time to establish connection #3134

John-LittleBearLabs commented Nov 17, 2022 •

edited

mxinden left a comment

mxinden left a comment

mxinden left a comment

mxinden left a comment

mergify bot commented Dec 12, 2022

mxinden left a comment

time to establish connection #3134

time to establish connection #3134

Conversation

John-LittleBearLabs commented Nov 17, 2022 • edited

Description

Notes

Links to any relevant issues

Open Questions

Change checklist

mxinden left a comment

Choose a reason for hiding this comment

mxinden left a comment

Choose a reason for hiding this comment

mxinden left a comment

Choose a reason for hiding this comment

mxinden left a comment

Choose a reason for hiding this comment

mergify bot commented Dec 12, 2022

mxinden left a comment

Choose a reason for hiding this comment

John-LittleBearLabs commented Nov 17, 2022 •

edited