Controller backend: reconcile each ntp in individual fibers #16055
Conversation
|
new failures in https://buildkite.com/redpanda/redpanda/builds/43647#018cf476-8100-43a4-9e7d-67a03af05ae4: |
|
Hmm, the above failure may actually be semi-related due to slightly changed timings of partition creation. In the test run ntp Looking at the code, I see that before the raft group manager is marked "ready", consensus instances are created with priority=1. When @mmaslankaprv WDYT? |
|
ok, this is very easy to reproduce with a well-placed sleep |
| auto [rs_it, inserted] = _states.try_emplace(d.ntp); | ||
| if (inserted) { | ||
| rs_it->second | ||
| = ss::make_lw_shared<ntp_reconciliation_state>(); | ||
| } |
|
Fix for the test_recovery_mode failure: #16068 |
Event is a simple notification mechanism. One fiber can set() the event and another can wait() for it.
ztlpn
force-pushed
the
controller-backend-fiber-per-ntp
branch
from
1e32c2b
to
fadddd2
Compare
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43737#018cffa6-0d05-4f85-8ffb-774fb45f569c |
In the future, when partitions will be assigned to shards by a node-level component (as opposed to topic table), controller backend will have to accept notifications from several sources. To support this it is more convenient to have per-ntp fibers doing reconciliation, as opposed to a single housekeeping fiber that periodically launches reconciliation for all pending ntps and waits for all reconciliation attempts to finish.
Backports Required
Release Notes