Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Profiling] Ensure responses are only sent once #93692

Merged

Conversation

danielmitterdorfer
Copy link
Member

For performance reasons the profiling plugin issues multiple requests concurrently. It uses internal handler classes to keep track of state. When all responses of dependent requests have arrived, it assembles the response and sends it to the client. The final state handler used two atomic counters to track outstanding requests. This has led to a race condition: When the two code paths finished at the same time, the plugin attempted to send the response twice.

With this commit we simplify state handling by only using a single atomic counter for both code paths. We decrement the counter only in a single place and will send the response iff all dependent requests have finished.

Closes #93691

For performance reasons the profiling plugin issues multiple requests
concurrently. It uses internal handler classes to keep track of state.
When all responses of dependent requests have arrived, it assembles the
response and sends it to the client. The final state handler used two
atomic counters to track outstanding requests. This has led to a race
condition: When the two code paths finished at the same time, the plugin
attempted to send the response twice.

With this commit we simplify state handling by only using a single
atomic counter for both code paths. We decrement the counter only in a
single place and will send the response iff all dependent requests have
finished.

Closes elastic#93691
@danielmitterdorfer danielmitterdorfer added >bug :Search/Search Search-related issues that do not fall into other categories v8.7.0 v8.7.1 v8.8.0 labels Feb 10, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Feb 10, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@elasticsearchmachine
Copy link
Collaborator

Hi @danielmitterdorfer, I've created a changelog YAML for you.

@danielmitterdorfer danielmitterdorfer added the auto-backport-and-merge Automatically create backport pull requests and merge when ready label Feb 10, 2023
mayFinish();
}

public void mayFinish() {
if (expectedStackFrameSlices.get() == 0 && expectedExecutableSlices.get() == 0) {
if (expectedSlices.decrementAndGet() == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@danielmitterdorfer
Copy link
Member Author

Thanks for the quick review @benwtrent!

@danielmitterdorfer danielmitterdorfer merged commit ce5b68e into elastic:main Feb 10, 2023
danielmitterdorfer added a commit to danielmitterdorfer/elasticsearch that referenced this pull request Feb 10, 2023
For performance reasons the profiling plugin issues multiple requests
concurrently. It uses internal handler classes to keep track of state.
When all responses of dependent requests have arrived, it assembles the
response and sends it to the client. The final state handler used two
atomic counters to track outstanding requests. This has led to a race
condition: When the two code paths finished at the same time, the plugin
attempted to send the response twice.

With this commit we simplify state handling by only using a single
atomic counter for both code paths. We decrement the counter only in a
single place and will send the response iff all dependent requests have
finished.

Closes elastic#93691
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.7

elasticsearchmachine pushed a commit that referenced this pull request Feb 10, 2023
For performance reasons the profiling plugin issues multiple requests
concurrently. It uses internal handler classes to keep track of state.
When all responses of dependent requests have arrived, it assembles the
response and sends it to the client. The final state handler used two
atomic counters to track outstanding requests. This has led to a race
condition: When the two code paths finished at the same time, the plugin
attempted to send the response twice.

With this commit we simplify state handling by only using a single
atomic counter for both code paths. We decrement the counter only in a
single place and will send the response iff all dependent requests have
finished.

Closes #93691
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport-and-merge Automatically create backport pull requests and merge when ready >bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team v8.7.0 v8.7.1 v8.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Profiling plugin may attempt to send response twice
3 participants