Timeout point send in forward_request verb comes from seastar::lowres_clock
#12458
Comments
|
@havaker please fix - it seams that you know exactly what the problem is 🙂 |
|
Note: this bug exists in 5.1 version (since the merge of #9209). |
Thanks for the notice. this can be backported when the fix is ready. We are using the milestones to reflect when this issue will have a fix (or at least when we intend to get to it) - therefore 5.2 is the correct milestone. |
This was referenced Jan 12, 2023
|
Fixed by bbbe12a |
avikivity
pushed a commit
that referenced
this issue
Jan 18, 2023
`forward_request` verb carried information about timeouts using `lowres_clock::time_point` (that came from local steady clock `seastar::lowres_clock`). The time point was produced on one node and later compared against other node `lowres_clock`. That behavior was wrong (`lowres_clock::time_point`s produced with different `lowres_clock`s cannot be compared) and could lead to delayed or premature timeout. To fix this issue, `lowres_clock::time_point` was replaced with `lowres_system_clock::time_point` in `forward_request` verb. Representation to which both time point types serialize is the same (64-bit integer denoting the count of elapsed nanoseconds), so it was possible to do an in-place switch of those types using logic suggested by @avikivity: - using steady_clock is just broken, so we aren't taking anything from users by breaking it further - once all nodes are upgraded, it magically starts to work Closes #12529 (cherry picked from commit bbbe12a) Fixes #12458
|
Backported to 5.1. 5.0 did not have the broken functionality. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the parallelized aggregation layer
query::forward_request(that is sent to remote nodes) carries information about timeouts usinglowres_clock::time_point(that came from localseastar::lowres_clock). Thelowres_clock::time_pointis later used on remote nodes to designate a timeout date. This is wrong and may lead to delayed or premature timeout.The text was updated successfully, but these errors were encountered: