[ISSUE #10651] Adapt RocketMQ to latest DLedger and remove fastjson1

[RongtongJin]

Which Issue(s) This PR Fixes

• Fixes [Enhancement] Upgrade DLedger to 0.3.3-SNAPSHOT #10651

Brief Description

This PR reworks the DLedger 0.3.3 adaptation on top of the latest DLedger code and openmessaging/dledger#336. It is intended to supersede the incomplete adaptation in #10650.

The change:

• adapts RocketMQ broker, store, and controller code to the latest DLedger APIs;
• restores immediate append positions required by RocketMQ through DLedger PR how to setDelayTimeLevel? #336;
• derives the committed physical boundary from DLedger index metadata and prevents reads from exposing an uncommitted tail;
• enables fast commit-index advancement and handles DLedger NOOP/control entries safely during append, reput, and abnormal recovery;
• handles latest DLedger's ordinary AppendFuture result for a single-message batch and BatchAppendFuture for multi-message batches;
• removes RocketMQ's fastjson1 dependency and aligns Maven/Bazel on fastjson2 2.0.64;
• adds JDK 8 cold-start compatibility fixtures, focused store regressions, and a real three-broker failover/full-restart integration test.

The validated compatibility boundary is a coordinated full-stop upgrade. Mixed-version rolling upgrade and downgrade are not covered by this PR.

Important

This is intentionally a draft. The current DLedger coordinate, 0.3.3-pr336-f2-64-SNAPSHOT, is a locally built artifact from DLedger master + PR #336 + fastjson2 2.0.64. Before this PR can be merged, DLedger #336 must be merged and a publicly resolvable DLedger artifact must be published; this PR must then switch to that coordinate and rerun the matrix.

Related work: #10650, #10928, openmessaging/dledger#336.

How Did You Test This Change?

All Java verification used Amazon Corretto JDK 8.

• DLedger PR how to setDelayTimeLevel? #336 regression: 6/6 tests passed; full DLedger reactor passed; RPC cold-start/network harness 3/3 passed.
• RocketMQ focused Maven tests:
  • remoting compatibility: 5/5;
  • DLedger store regression suite: 8/8;
  • controller: 7/7;
  • broker DLedger integration tests: 2/2;
  • broker container DLedger lifecycle: 1/1.
• RocketMQ full Maven clean test: 19/19 modules, 3,124 tests, 0 failures, 0 errors, 42 skipped.
• Bazel affected build: 5 targets, 203 actions.
• Bazel tests: remoting 5/5, store 8/8, controller 1/1, single-broker IT 1/1, three-broker IT 1/1.
• Full-stop data compatibility fixture: DLedger 0.3.2 wrote 10 messages; latest code read all 10 before the first new append, then wrote/read message 11.
• Final dependency/bytecode audit: 7 packaged JARs and 1,360 classes scanned; no fastjson1 dependency, JAR entry, or class constant remained.
• git diff --check passed.

[fuyou001]

LGTM

[RockteMQ-AI]

Summary

This PR adapts RocketMQ to the latest DLedger API (0.4.3) and removes the fastjson1 dependency in favor of fastjson2 2.0.64. The changes are well-structured and address a critical adaptation need.

Strengths:

• Clean API migration (package reorganization, getCommittedIndex() moved to MemberState)
• Proper committed boundary tracking with getCommittedPos() caching prevents reading uncommitted data
• Good NOOP/control entry handling during append, reput, and recovery
• Comprehensive test coverage including a 601-line three-broker failover IT
• fastjson1→fastjson2 migration with backward compatibility fixtures (RemotingSerializableCompatTest)

Notes:

• The PR description mentions this is "intentionally a draft" pending DLedger PR #336 merge, but the PR is marked as ready for review on GitHub. Please confirm the intended status.
• The calculate-coverage CI check is failing — this may need attention before merge.
• The WORKSPACE file references dledger:0.4.3 while the PR body mentions 0.3.3-pr336-f2-64-SNAPSHOT. Please clarify which version is the target.

Overall the code changes look solid and the adaptation is thorough. The test coverage is particularly impressive.

---

Automated review by "github-manager-bot"

Findings

• [Info] DLedgerCommitLog.java:75 — The cachedCommittedPos/cachedCommittedIndex caching pattern is well-designed. Consider adding a brief Javadoc explaining the cache invalidation contract.

• [Info] DLedgerCommitLog.java:247 — firstNormalEntryBody() correctly skips NOOP entries during recovery. Good defensive handling of all-NOOP mapped files.

• [Warning] DLedgerCommitLog.java:500 — In getMessage(), the guard committedPos < 0 means no entry committed yet. Consider logging at DEBUG level when this triggers to help diagnose startup timing issues.

• [Info] DefaultMessageStore.java:2741 — The reput guard dispatchRequest.getMsgSize() > 0 correctly prevents dispatching empty messages from NOOP/control entries.

[RockteMQ-AI]

⚠️ Merge conflict detected

This PR has conflicts with the develop branch and cannot be merged. Please rebase or merge develop into your branch and resolve the conflicts:

git fetch origin
git checkout codex/dledger-latest-pr336-adapter
git rebase origin/develop
# resolve conflicts, then:
git push --force-with-lease

This is a one-time reminder. Feel free to @mention me for a re-review after conflicts are resolved.

---

Automated notification by "github-manager-bot"