perf: deserialize DKG messages once (framing-only intake) - #7557
perf: deserialize DKG messages once (framing-only intake)#7557PastaPastaPasta wants to merge 3 commits into
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
Next review available in: 4 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
Comment |
|
⛔ Blockers found — Opus deferred (commit 11cd6c4) |
Potential PR merge conflictsThis is advisory only. It does not block CI, but it marks PRs that will likely need a rebase depending on merge order. If this PR merges firstThese open PRs will likely need a rebase:
|
783af2b to
8389144
Compare
DKG intake previously deserialized a copy of each accepted payload (repeating BLS point decompression on the shared network thread) for structural validation, and the DKG worker then deserialized the retained bytes again. Replace the typed intake pass with a framing-only wire walk that validates CompactSize counts, dynamic bitsets (via the same ReadFixedBitSet the typed path uses), quorum-parameter bounds, truncation, and trailing bytes without decoding any BLS object. The worker is now the sole typed deserialization point, immediately followed by the same parameter-derived structural checks. The pre-existing per-peer pending-message quota is rekeyed from NodeId to the MNAuth-verified proTxHash and made cumulative for the round, so a sender can no longer reset its retention budget by reconnecting or by waiting for the worker to drain the queue. Own messages are enqueued under this node's own proTxHash and share the same quota path. Sender identities are pinned to the deterministic masternode list by MNAuth, so worst-case retention is bounded by (hostile MN count) x quota. Duplicate hashes are rejected before charging the quota, and quota-dropped messages are not marked seen so another peer with budget can re-deliver them. The llmqType/quorumHash prefix is peeked via SpanReader instead of read+Rewind, and short payloads are scored instead of throwing out of ProcessMessage. Leftover raw queues are discarded at round start without BLS work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Intake framing validation and worker typed deserialization are two hand-maintained parsers over one wire format. The safety-critical direction is that framing must never reject a payload the worker would accept, otherwise honest DKG messages are silently dropped before retention and quorum formation degrades. Assert that direction over fuzzer-provided payloads for every configured LLMQ and both BLS schemes, plus a constructed well-formed message per input so serializer/framing drift is caught even from an empty corpus. The converse is intentionally not asserted: framing accepts undecodable BLS encodings so the worker can score the sender. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Unit tests pin the CDKGPendingMessages semantics: the per-proTx quota survives reconnects and is not refunded by drains, duplicates are rejected before charging, quotas are independent across proTxes, and own messages are charged under this node's own proTxHash. Functional tests cover trailing-byte rejection at intake, deferral of BLS decoding to the DKG worker (scored there, not at intake), quota persistence across reconnects under fresh NodeIds, and late-message retention cleared at round start without BLS work. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
8389144 to
11cd6c4
Compare
thepastaclaw
left a comment
There was a problem hiding this comment.
Preliminary review — Codex only
The framing parser and proTxHash-keyed quota redesign are well covered, but moving malformed BLS detection to the DKG worker creates a deterministic penalty bypass when the originating peer disconnects before its queued message is processed, so changes are required. The new Dash-specific fuzz target must also be registered in the non-backported manifest to receive the intended lint coverage.
Source: Reviewer backend models: codex general gpt-5.6-sol and codex dash-core-commit-history gpt-5.6-sol; final verifier backend model: gpt-5.6-sol. openclaw-agent/cliproxy/gpt-5.6-sol is orchestration-only and not reviewer evidence.
Validated blockers were found in the Codex precheck. Opus is deferred until a fresh Codex revalidation clears the blocker gate.
Review provenance
- Codex reviewers:
gpt-5.6-sol— general (completed),gpt-5.6-sol— dash-core-commit-history (completed) - Verifier:
gpt-5.6-sol— verifier - Sonnet: not run (deferred by blocker gate)
🔴 1 blocking | 🟡 1 suggestion(s)
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `src/llmq/net_dkg.cpp`:
- [BLOCKING] src/llmq/net_dkg.cpp:403-407: Deferred BLS failures lose their penalty after reconnect
The queued entry retains only the originating `NodeId`, while this PR moves malformed BLS detection from synchronous intake validation to the later DKG worker pass. A peer can send a requested, framing-valid payload with an invalid BLS encoding during `Initialized`, disconnect before the matching phase drains the queue, and reconnect under a new ID. `FinalizeNode()` removes the old ID from `PeerManagerImpl::m_peer_map`, so the later `PeerMisbehaving(nodeId, 100)` call finds no `PeerRef` and silently applies no score. This is a regression from the previous typed intake check, which detected this malformed encoding while the sender was still being processed. The proTxHash-keyed quota limits retained work but does not preserve punishment; retain enough authenticated sender metadata to apply the offense after disconnect, or otherwise ensure deferred validation keeps the originating peer punishable.
In `src/test/fuzz/dkg_message_framing.cpp`:
- [SUGGESTION] src/test/fuzz/dkg_message_framing.cpp:1: Register the new Dash-specific fuzz file as non-backported
`src/test/fuzz/dkg_message_framing.cpp` is a newly added Dash-specific source, but no pattern in `test/util/data/non-backported.txt` matches it; `src/test/llmq*.cpp` only covers files directly under `src/test`. The manifest feeds Dash-specific cppcheck and clang-diff-format coverage, so add the fuzz target's exact path to it.
| for (const auto& p : msgs) { | ||
| const NodeId& nodeId = p.first; | ||
| if (!p.second) { | ||
| LogPrint(BCLog::LLMQ_DKG, "%s -- failed to deserialize message, peer=%d\n", __func__, nodeId); | ||
| auto msg = DeserializeCheckedDKGMessage<Message>(*p.second, params, nodeId); | ||
| if (!msg) { | ||
| peerman.PeerMisbehaving(nodeId, 100); |
There was a problem hiding this comment.
🔴 Blocking: Deferred BLS failures lose their penalty after reconnect
The queued entry retains only the originating NodeId, while this PR moves malformed BLS detection from synchronous intake validation to the later DKG worker pass. A peer can send a requested, framing-valid payload with an invalid BLS encoding during Initialized, disconnect before the matching phase drains the queue, and reconnect under a new ID. FinalizeNode() removes the old ID from PeerManagerImpl::m_peer_map, so the later PeerMisbehaving(nodeId, 100) call finds no PeerRef and silently applies no score. This is a regression from the previous typed intake check, which detected this malformed encoding while the sender was still being processed. The proTxHash-keyed quota limits retained work but does not preserve punishment; retain enough authenticated sender metadata to apply the offense after disconnect, or otherwise ensure deferred validation keeps the originating peer punishable.
source: ['codex']
| @@ -0,0 +1,157 @@ | |||
| // Copyright (c) 2025 The Dash Core developers | |||
There was a problem hiding this comment.
🟡 Suggestion: Register the new Dash-specific fuzz file as non-backported
src/test/fuzz/dkg_message_framing.cpp is a newly added Dash-specific source, but no pattern in test/util/data/non-backported.txt matches it; src/test/llmq*.cpp only covers files directly under src/test. The manifest feeds Dash-specific cppcheck and clang-diff-format coverage, so add the fuzz target's exact path to it.
source: ['codex']
Issue being fixed or feature implemented
DKG network message intake deserializes each accepted payload twice: once on a copied payload in the p2p message handler for structural validation, and again from the pending queue on the DKG worker thread. These messages carry BLS objects, so the redundant intake pass repeats elliptic-curve point decompression on the shared network thread during every DKG round.
Additionally, the per-peer pending-message quota in the same path is keyed by
NodeId, so a single misbehaving masternode can reset its retention budget just by reconnecting, defeating the bound that already exists.This is a from-scratch redesign of the approach in #7401, sharing its goals and test strategy but with a substantially smaller intake parser and a simpler retention model.
What was done?
CheckDKGMessageWireStructure(), a framing-only walk that validates CompactSize counts, dynamic bitsets, quorum-parameter bounds, truncation, and trailing bytes without decoding any BLS object. The walk is deliberately thin: fixed-size BLS encodings are skipped with a single bounds-checkedignore(), and dynamic bitsets are validated by calling the sameReadFixedBitSet()the typed deserializer uses, so truncation and padding-bit semantics cannot diverge.llmqType/quorumHashprefix is peeked viaSpanReaderinstead of read-then-Rewind, and short payloads are rejected up front instead of throwing out ofProcessMessage.maxMessagesPerProTx) instead ofNodeId, so reconnecting with a freshNodeIdno longer resets the budget, and the quota is cumulative for the round (draining the queue does not refund it). Own messages are enqueued under this node's own proTxHash and go through the same quota path -- no special case.payload || garbagewas accepted and hashed as a distinct inventory item.dkg_message_framing) that continuously checks the safety-critical equivalence direction across every configured LLMQ and both BLS schemes: the framing walk must never reject a payload that typed worker deserialization would accept. The converse is intentionally not asserted -- framing accepts undecodable BLS encodings so the worker can score the sender.How Has This Been Tested?
Validated on macOS arm64 with:
Breaking Changes
None.
Checklist: