Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add incoming/outgoing connections metrics #959

Merged
merged 7 commits into from
Apr 8, 2022
Merged

Add incoming/outgoing connections metrics #959

merged 7 commits into from
Apr 8, 2022

Conversation

mcamou
Copy link
Contributor

@mcamou mcamou commented Apr 6, 2022

No description provided.

@@ -38,6 +39,30 @@ const (
ReshareShutdown ReshareStatus = "node_stopped"
)

type statsHandler struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this whole type is an IncomingConnectionsStatsHandler because it tags to the IncomingConnectionTimestamp metric.

Consider moving this statsHandler struct over to the net package, so that when e.g. the http client package pulls in metrics, it doesn't take on the grpc dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call, thanks! Fixed.

@mcamou mcamou self-assigned this Apr 6, 2022
@mcamou mcamou added the monitoring Area: Monitoring label Apr 6, 2022
@mcamou mcamou added this to the Ceremony dashboard milestone Apr 6, 2022
@mcamou mcamou force-pushed the metrics branch 3 times, most recently from 368aa5f to 268ebe6 Compare April 7, 2022 09:24
@mcamou
Copy link
Contributor Author

mcamou commented Apr 7, 2022

@willscott @AnomalRoil I would appreciate some help regarding why test_chained and test_unchained are failing

@@ -311,9 +319,16 @@ func (g *grpcClient) conn(p Peer) (*grpc.ClientConn, error) {
metrics.GroupDialFailures.WithLabelValues(p.Address()).Inc()
}
}
g.conns[p.Address()] = c
metrics.GroupConnections.Set(float64(len(g.conns)))
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand where err is supposedly set now?

I think this is never run with the current code when ok == true

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, and in the middle of solving this I figured out that we don't want to have the state as a label. I've fixed this and added a couple of comments explaining my reasoning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, the err = nil should not be necessary, but it's never a bad thing to be explicit:

gore version 0.5.3  :help for help
gore> var err error
gore> err
nil

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is unnecessary and doing the converse of what you want, no?

net/client_grpc.go Outdated Show resolved Hide resolved
@@ -311,9 +319,16 @@ func (g *grpcClient) conn(p Peer) (*grpc.ClientConn, error) {
metrics.GroupDialFailures.WithLabelValues(p.Address()).Inc()
}
}
g.conns[p.Address()] = c
metrics.GroupConnections.Set(float64(len(g.conns)))
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is unnecessary and doing the converse of what you want, no?

}

// Emit the connection state regardless of whether it's a new or an existing connection
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you check if there was an error before firing the metric?
Why not fire it all the time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

err will be non-nil only if grpc.Dial has an error, in which case c might be nil or unset. In that case, c.GetState() would bomb out.

Copy link
Contributor Author

@mcamou mcamou Apr 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and yes, you're right above re: err != nil. Fixing both.

@mcamou mcamou merged commit 185a337 into master Apr 8, 2022
@mcamou mcamou deleted the metrics branch April 8, 2022 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
monitoring Area: Monitoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants