Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the scope current when needed for grpc server instrumentation #3806

Conversation

ttddyy
Copy link
Contributor

@ttddyy ttddyy commented May 3, 2023

When grpc client uses non-blocking requests, the server calls interceptor#interceptCall multiple times first, then proceed to the listener#on... methods. It is different from blocking requests that call those methods sequentially per request. This behavior causes multiple scopes to be opened on the same thread in interceptCall and wrongly consumes them in on... methods.

This change compares the current scope held by the ObservationRegistry against the withheld scope by the grpc listener callback. When they are different, make the withheld scope current; then, the subsequent logic can pick up the proper target observation.

Closes gh-3805

When grpc client uses non-blocking requests, the server calls
`interceptor#interceptCall` multiple times first, then proceed to the
`listener#on...` methods.  It is different from blocking requests that call
those methods sequentially per request.  The behavior causes multiple scopes to
be opened on the same thread in `interceptCall` and wrongly consumes them in
`on...` methods.

This change compares the current scope held by the `ObservationRegistry`
against the withheld `scope`.  When they are different, make the withheld scope
current; then, the subsequent logic can pick the proper target observation.

Closes micrometer-metricsgh-3805
@ttddyy ttddyy changed the base branch from main to 1.10.x May 3, 2023 06:19
@jonatan-ivanov jonatan-ivanov added this to the 1.10.7 milestone May 3, 2023
@jonatan-ivanov jonatan-ivanov added bug A general bug instrumentation An issue that is related to instrumenting a component module: micrometer-binders labels May 3, 2023
}

@Override
public void onMessage(RespT message) {
if (this.scope != this.registry.getCurrentObservationScope()) {
this.scope.makeCurrent();
this.registry.setCurrentObservationScope(this.scope);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Afair this is redundant cause scope will make itself current.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, makeCurrent does not update the registry.getCurrentObservationScope().
Created a PR to fix it: #3808

listenerA.onHalfClose();
assertThat(registry.getCurrentObservation()).isSameAs(observationA);
assertThat(registry.getCurrentObservationScope()).isSameAs(scopeA);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test to ensure that we're not leaving any unclosed scopes? I understand that after making all calls current scope should be null?

@shakuzen shakuzen modified the milestones: 1.10.7, 1.10.8 May 9, 2023
@shakuzen shakuzen added module: micrometer-core An issue that is related to our core module and removed module: micrometer-binders labels May 9, 2023
@ttddyy
Copy link
Contributor Author

ttddyy commented May 14, 2023

superseded by #3836

@ttddyy ttddyy closed this May 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A general bug instrumentation An issue that is related to instrumenting a component module: micrometer-core An issue that is related to our core module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GRPC current span Tracer.currentSpan() becomes null when processing parallel calls
4 participants