Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Processing requests in order in Kafka layer #1496

Merged
merged 4 commits into from
Jun 15, 2021
Merged

Processing requests in order in Kafka layer #1496

merged 4 commits into from
Jun 15, 2021

Commits on Jun 15, 2021

  1. k/response: introduced process_result_stages type 

    Introduced a type aggregating futures representing two stages of kafka
request processing. This way request handler will be able to decide
which part of the processing should be executed in foreground (blocking
other requests from being handled) and which can be executed in
background asynchronous to other requests processing.

Signed-off-by: Michal Maslanka <michal@vectorized.io>
    @mmaslankaprv
    mmaslankaprv committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    711685f View commit details
    Browse the repository at this point in the history

  2. k/requests: split execution of request processing in two stages 

    Split handling of Kafka request into two stages. Dispatch stage is
executed in foreground while the second stage is executed in background.
This way we can leverage the fact that request processing order is
established before its processing completely finished and handle
multiple requests at the time without compromising correct ordering.

Signed-off-by: Michal Maslanka <michal@vectorized.io>
    @mmaslankaprv
    mmaslankaprv committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    7412536 View commit details
    Browse the repository at this point in the history

  3. k/produce: handling produce request in two stages 

    Implemented two stage handling of produce request. The two phases of
produce request processing are reflected in two phases of
`cluster::partition::replicate` this way redpanda can handle multiple
requests per connection while still not changing the request processing
ordering.

Signed-off-by: Michal Maslanka <michal@vectorized.io>
    @mmaslankaprv
    mmaslankaprv committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    2750910 View commit details
    Browse the repository at this point in the history

  4. k/offset_commit: processing offset commit request in two stages 

    Offsets commit handler uses raft to replicate offset commit requests.
Leveraging raft two stage replicate processing to to handle multiple
in-flight offset commit requests and prevent contention.

Signed-off-by: Michal Maslanka <michal@vectorized.io>
    @mmaslankaprv
    mmaslankaprv committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    d0fab2c View commit details
    Browse the repository at this point in the history