op-node: RPC Limit client does not respect context #7525
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change addresses two bugs in the limit wrapper for client RPCs.
The existing
limit.go
implements its own semaphore which ignores the passed in context. This means that the semaphore will block indefinitely, even when the context of the request has already expired.The existing implementation does not guard against clients which invoke RPC methods after the channel underlying the custom semaphore has been closed. This results in panics where the closed channel is written to during shutdown, and results in test flakiness. This flakiness is most evident in the op-e2e-http-tests suite.
Along with these fixes comes a test which attempts to demonstrate the previous bad behavior. Because these bugs are inherently tied to the interaction of multiple go routines, the test ends up being a bit complex, but is well commented and hopefully remains readable.