[ISSUE #8408]♻️Coordinate topic persistence and registration shutdown

[mxsm]

Which Issue(s) This PR Fixes(Closes)

• Fixes [Refactor♻️] Coordinate topic persistence and registration shutdown #8408

Brief Description

• Add a single leased FIFO coordinator for topic configuration persistence and broker registration.
• Route topic metadata file and RocksDB I/O through the injected blocking executor, drain accepted work before unregistering, and expose a typed shutdown report.
• Atomically replace RocksDB topic snapshots with their data version and close shared broker configuration backends only through the aggregate owner.
• Update the architecture migration checklist and reviewed ArcMut baseline for the audited one-to-one occurrence relocation.

How Did You Test This Change?

• cargo check -p rocketmq-broker --all-targets --all-features
• cargo clippy -p rocketmq-broker --all-targets --all-features -- -D warnings
• cargo test -p rocketmq-store --features rocksdb_store --test rocksdb_foundation_tests
• cargo test -p rocketmq-store --features rocksdb_store --test rocksdb_store_semantics_tests
• cargo test -p rocketmq-broker --features rocksdb_store rocksdb
• cargo test -p rocketmq-broker --features rocksdb_store pop_consumer
• cargo test -p rocketmq-broker --all-features --lib -- --test-threads=1 (592 passed, 25 pre-existing baseline failures, 1 ignored; no new baseline failures)
• .\scripts\runtime-audit.ps1 -SkipBaseline -EnforceBoundaryBaseline
• python scripts/arc_mut_guard.py and python scripts/arc_mut_guard.py --fixtures
• Architecture dependency, release, performance, and guard test suites
• cargo fmt --all -- --check
• cargo clippy --workspace --no-deps --all-targets --all-features -- -D warnings

Summary by CodeRabbit

• New Features

  • Added coordinated topic configuration persistence and broker registration.
  • Improved topic creation, updates, deletions, and retry-topic setup with reliable asynchronous persistence.
  • Added atomic configuration snapshot updates, including stale-entry removal and version tracking.
  • Enhanced shutdown reporting with topic configuration health, timeout, and pending-work status.
  • Added asynchronous topic configuration export and metadata loading.

• Documentation

  • Updated architecture and migration progress documentation to reflect completed coordinator work and remaining milestones.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 586535b6-e87e-4d96-b5c2-9a09e45e6e2b

📥 Commits

Reviewing files that changed from the base of the PR and between f87b8ab and fd64968.

📒 Files selected for processing (17)

• docs/plans/architecture-refactor-migration/CHECKLIST.md
• docs/plans/architecture-refactor-migration/README.md
• docs/plans/architecture-refactor-migration/phase-3-production-readiness/11-security-observability-cloud.md
• docs/plans/architecture-refactor-migration/phase-3-production-readiness/11-soundness-closure-progress.md
• rocketmq-broker/src/broker_runtime.rs
• rocketmq-broker/src/config.rs
• rocketmq-broker/src/config/rocksdb_manager.rs
• rocketmq-broker/src/offset/manager/consumer_offset_manager.rs
• rocketmq-broker/src/processor/admin_broker_processor/broker_config_request_handler.rs
• rocketmq-broker/src/processor/admin_broker_processor/topic_request_handler.rs
• rocketmq-broker/src/processor/processor_service/pop_revive_service.rs
• rocketmq-broker/src/slave/slave_synchronize.rs
• rocketmq-broker/src/subscription/manager/subscription_group_manager.rs
• rocketmq-broker/src/topic/manager.rs
• rocketmq-broker/src/topic/manager/topic_config_coordinator.rs
• rocketmq-broker/src/topic/manager/topic_config_manager.rs
• scripts/arc-mut-baseline.json

---

Walkthrough

The broker adds a TopicConfigCoordinator for serialized topic persistence and registration, coordinated shutdown draining, atomic RocksDB snapshot replacement, and broker shutdown reporting. Topic mutation, registration, metadata loading, replication, admin export, tests, and migration documentation are updated accordingly.

Changes

Topic persistence foundation

Layer / File(s)	Summary
Snapshot persistence and RocksDB replacement rocketmq-broker/src/topic/manager/topic_config_manager.rs, rocketmq-broker/src/config/rocksdb_manager.rs, rocketmq-broker/src/config.rs	Topic updates now persist latest snapshots, while RocksDB atomically replaces rows, removes stale keys, and stores DataVersion; tests cover replacement and restart behavior.

Coordinator execution

Layer / File(s)	Summary
Coordinator worker and shutdown barriers rocketmq-broker/src/topic/manager/topic_config_coordinator.rs, rocketmq-broker/src/topic/manager.rs	A bounded coordinator serializes persistence and registration, routes blocking I/O through the executor, tracks failures, drains accepted work, and reports shutdown state.

Broker integration

Layer / File(s)	Summary
Broker runtime wiring and shutdown integration rocketmq-broker/src/broker_runtime.rs, scripts/arc-mut-baseline.json	Broker runtime construction, metadata loading, full registration, shutdown reporting, shared RocksDB closure, and related tests use the coordinator.
Mutation, replication, and admin call sites rocketmq-broker/src/processor/..., rocketmq-broker/src/slave/..., rocketmq-broker/src/offset/..., rocketmq-broker/src/subscription/...	Topic changes, retry-topic creation, slave synchronization, exports, metrics, and manager shutdown paths use coordinator APIs.

Migration tracking

Layer / File(s)	Summary
Migration documentation and verification docs/plans/architecture-refactor-migration/...	M11-12bc2 completion, validation evidence, coordinator behavior, ArcMut accounting, and the next migration slice are documented.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

• #8406 — Covers the same topic metadata ownership, persistence, and migration paths.
• #8404 — Overlaps with the broker registration flow and ArcMut baseline update.
• #8292 — Relates to ArcMut governance and the shared baseline entry.

Possibly related PRs

• mxsm/rocketmq-rust#7535 — Related Broker shutdown component reporting.
• mxsm/rocketmq-rust#7789 — Related slave topic-configuration persistence flow.
• mxsm/rocketmq-rust#8405 — Related broker registration and ArcMut baseline changes.

Suggested labels: refactor♻️, enhancement⚡️, approved, auto merge, AI review first

Suggested reviewers: rocketmq-rust-bot, teslarustor

Poem

A rabbit guards the topic queue,
While snapshots hop in order true.
RocksDB sheds its stale old key,
The shutdown gate waits patiently.
“Persist, register, then rest!”
Whiskers cheer the coordinator’s quest.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch mxsm/architecture-refactor-topic-config-coordinator

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[rocketmq-rust-bot]

🔊@mxsm 🚀Thanks for your contribution🎉！

💡CodeRabbit(AI) will review your code first🔥！

Note

🚨The code review suggestions from CodeRabbit are to be used as a reference only, and the PR submitter can decide whether to make changes based on their own judgment. Ultimately, the project management personnel will conduct the final code review💥.