bench: add Share Consumer (KIP-932) + KPipe KEY_ORDERED arms#138
Merged
Conversation
Extend ParallelProcessingBenchmark with two new runtimes so the competitive suite can answer "share consumer vs key-ordered dispatch," reusing the existing Testcontainers harness so every variable except the consumption mechanism is pinned (same broker, seed, workMicros sweep, and @OperationsPerInvocation throughput metric). - Share Consumer arm: one KafkaShareConsumer, each polled batch fanned to virtual threads with a per-batch barrier before the next poll (implicit ACCEPT = at-least-once). This is the controlled apples-to-apples rival of KPipe PARALLEL — both unordered, one process, VT fan-out; only the fetch+acknowledge protocol differs. - KPipe KEY_ORDERED arm: same single consumer, per-key serial queues. The only arm with an ordering guarantee — present to quantify the per-key ordering tax, NOT as a head-to-head (no other runtime, share included, can order). - Keyed seeding (KEY_CARDINALITY=1000) so KEY_ORDERED is meaningful; unordered arms ignore keys. - Broker env for share groups (share rebalance protocol + __share_group_state topic RF) and a per-group share.auto.offset.reset=earliest via Admin so the fresh share group reads pre-seeded records. - METHODOLOGY.md documents the apples-to-apples framing, the ordering-tax caveat, the single-process (not scale-out) scope, and the share-group setup as a verify-on-first-Docker-run item. Compiles against the 4.2.0 client API; NOT run (no Docker in the authoring environment), so the share-group runtime path is unvalidated — see the METHODOLOGY caveat.
…ound A smoke run on Testcontainers Kafka 4.2.0 confirmed the share group forms, assigns all 8 partitions, and consumes the seeded records (so the earliest offset reset took). Replace the "unvalidated / not run" caveat with the validated state, and record that the share arm's throughput is ~flat across workMicros — protocol/ack-bound, not work-bound — so it needs tuning before its numbers are quoted.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #138 +/- ##
=======================================
Coverage ? 74.64%
Complexity ? 763
=======================================
Files ? 76
Lines ? 2930
Branches ? 367
=======================================
Hits ? 2187
Misses ? 568
Partials ? 175 ☔ View full report in Codecov by Sentry. |
First full publishing run (fork=2, warmup=3, iter=5, gc profiler) on the 6-arm ParallelProcessingBenchmark. Headlines (Intel i7-8850H, JDK 25, Testcontainers Kafka 4.2.0; records/sec, see the file for error bars): - Raw VT ~630k and KPipe PARALLEL ~525–581k stay flat across workMicros — Loom absorbs blocking work; KPipe tracks the raw baseline within ~10%. - KPipe KEY_ORDERED shows the ordering tax: 312k → 90k as work rises, still ahead of Confluent/Reactor at 1000us. - Reactor collapses 310k → 9k under blocking work (core-bound scheduler); Confluent is pool-bound at ~64k. - Share consumer is protocol-bound and flat at ~4.8k regardless of work — but this is a SINGLE share member; the bench does not test share's scale-out, so it's a single-member ceiling, not a verdict. Raw JMH JSON committed alongside. Latency + batch-sink sections marked N/A (only ParallelProcessingBenchmark was run). METHODOLOGY table re-aligned by spotless.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two runtimes to the competitive
ParallelProcessingBenchmark, building on the now-merged key-ordered work (#137):KafkaShareConsumer; each polled batch is fanned to virtual threads with a per-batch barrier before the next poll (implicitACCEPT= at-least-once). The controlled apples-to-apples rival of KPipe PARALLEL: both unordered, one process, VT fan-out — only the fetch+acknowledge protocol differs.Both reuse the existing Testcontainers harness, so every variable except the consumption mechanism is pinned: same broker, seed,
workMicrossweep, and@OperationsPerInvocationthroughput metric.Supporting changes:
KEY_CARDINALITY=1000) so KEY_ORDERED is meaningful; the unordered arms ignore keys.__share_group_statetopic RF) and a per-groupshare.auto.offset.reset=earliestset viaAdmin, so the fresh share group reads the pre-seeded records.METHODOLOGY.md: the apples-to-apples framing, the ordering-tax caveat, and the single-process (not scale-out) scope.Published results
Full publishing run committed in this PR:
benchmarks/results/2026-05-29.md+ rawresults-2026-05-29.json. fork=2, warmup=3, iter=5,gcprofiler. Intel i7-8850H (6c/12t), JDK 25.0.2, Testcontainers Kafka 4.2.0.Throughput (records/s; see the results file for ± error bars):
KafkaConsumer+ VTTakeaways (full detail in the results file):
workMicros. This is a single share member — the bench does not exercise share's scale-out (members > partitions), so read it as a single-member ceiling in early KIP-932, not "share is slow," and not an ordering verdict against KEY_ORDERED (they aren't rivals).Caveats (also in the results file): Intel x86 not Apple Silicon (don't compare absolute numbers across machines);
workMicros=0is the noisiest cell; macOS + Docker loopback broker means absolute records/s are low — read orderings, not absolutes.Test plan
max.poll.records, ack batching, acquisition-lock timeout).