fix(persistence): bound RDB stream/count allocations against untrusted lengths#269
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
📝 WalkthroughWalkthroughRDB ChangesRDB stream hardening
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…d lengths PR #266 fixed the classic untrusted-length allocation-DoS pattern (a tiny crafted file claiming a huge count, driving a multi-GB Vec::with_capacity before a byte of the claimed data is verified) in snapshot.rs and redis_rdb.rs, and established the guard: call `rdb::validate_count` against the cursor's remaining bytes before any count-driven allocation. rdb.rs's `read_entry` / `read_entry_zero_copy` already applied this to every count in the HASH/LIST/SET/SORTED_SET branches and to the stream's `entry_count` and per-entry `field_count` — but four counts in the TYPE_STREAM consumer-group section were left unguarded: `group_count`, `pel_count`, `consumer_count`, and `pending_count`. None of the four currently drives an eager `with_capacity`/zero-fill (they grow a BTreeMap/HashMap incrementally, so today's worst case is bounded by actual bytes consumed, not amplified), but that's a structural inconsistency with every sibling count in the same function and a latent trap for the next refactor that adds a preallocated collection here. Fixed by adding the same validate_count guard, with the true structural minimum bytes per item so no legitimate file is ever rejected: - group_count: 28 bytes/group (4-byte empty name len + 16-byte last_delivered_id + 4-byte pel_count + 4-byte consumer_count) - pel_count: 36 bytes/entry (16-byte StreamId + 4-byte empty consumer name len + 16-byte delivery_time/delivery_count) - consumer_count: 16 bytes/consumer (4-byte empty name len + 8-byte seen_time + 4-byte pending_count) - pending_count: 16 bytes/id (StreamId) Applied identically to both `read_entry` (used by `load_from_bytes`, the AOF-preamble path) and `read_entry_zero_copy` (used by `load`, the boot-time RDB path) — the two are literal duplicates for this branch. Red/green TDD: added test_stream_{group,pel,consumer,pending}_count_dos_ rejected, each a crafted TYPE_STREAM value body with one lying count and zero trailing bytes to back it, confirmed rejected by both read paths. Added test_stream_consumer_group_round_trip_not_rejected as a control: a fully populated 1-group/1-pel/1-consumer/1-pending stream must still be accepted by both functions, proving the chosen minimums never reject a valid file. cargo test --release --lib persistence::rdb: 18/18 passing. cargo clippy -D warnings clean on both default features and --no-default-features --features runtime-tokio,jemalloc. cargo fmt clean. author: Tin Dang <tindang.ht97@gmail.com>
114208a to
b5af8ad
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/persistence/rdb.rs (2)
2033-2035: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider verifying parsed contents in the zero-copy control assertion.
The
read_entrypath in the control test verifies group count, PEL length, consumer count, and pending count. The zero-copy path only assertsis_ok(), so a parsing bug that returnsOkwith wrong data would go undetected.♻️ Suggested: mirror the read_entry assertions for the zero-copy path
// Same body must also be accepted by the zero-copy variant. let mut cursor2 = Cursor::new(&entry_bytes[..]); - assert!(read_entry_zero_copy(&mut cursor2, TYPE_STREAM, 0, false).is_ok()); + let (_key2, entry2) = read_entry_zero_copy(&mut cursor2, TYPE_STREAM, 0, false) + .expect("legit stream must not be rejected by zero-copy path"); + match entry2.value.as_redis_value() { + RedisValueRef::Stream(stream) => { + assert_eq!(stream.groups.len(), 1); + let group = stream.groups.get(&Bytes::from_static(b"g")).unwrap(); + assert_eq!(group.pel.len(), 1); + assert_eq!(group.consumers.len(), 1); + assert_eq!( + group.consumers.get(&Bytes::from_static(b"c")).unwrap().pending.len(), + 1 + ); + } + _ => panic!("Expected Stream"), + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/persistence/rdb.rs` around lines 2033 - 2035, Update the zero-copy control assertion using read_entry_zero_copy to retain and inspect the parsed result, then mirror the existing read_entry checks for group count, PEL length, consumer count, and pending count instead of asserting only is_ok().
1979-2036: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoffVerify file length against the 1500-line guideline.
This file appears to exceed 1500 lines (tests end at line 2037). As per coding guidelines, no single
.rsfile should exceed 1500 lines; larger modules should be split into submodules. Consider extracting the test module into a separate file (e.g.,rdb_tests.rsortests/rdb.rs).As per coding guidelines: "No single
.rsfile should exceed 1500 lines; split larger modules into submodules."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/persistence/rdb.rs` around lines 1979 - 2036, src/persistence/rdb.rs exceeds the 1500-line guideline because its tests are embedded in the implementation file. Extract the test module, including test_stream_consumer_group_round_trip_not_rejected and related tests, into a dedicated submodule such as rdb_tests.rs or tests/rdb.rs, then wire it into the module with the appropriate cfg(test) declaration and preserve access to required helpers and private items.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/persistence/rdb.rs`:
- Around line 2033-2035: Update the zero-copy control assertion using
read_entry_zero_copy to retain and inspect the parsed result, then mirror the
existing read_entry checks for group count, PEL length, consumer count, and
pending count instead of asserting only is_ok().
- Around line 1979-2036: src/persistence/rdb.rs exceeds the 1500-line guideline
because its tests are embedded in the implementation file. Extract the test
module, including test_stream_consumer_group_round_trip_not_rejected and related
tests, into a dedicated submodule such as rdb_tests.rs or tests/rdb.rs, then
wire it into the module with the appropriate cfg(test) declaration and preserve
access to required helpers and private items.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b8574f98-5d12-442a-94cd-de11cd6cbf4b
📒 Files selected for processing (2)
CHANGELOG.mdsrc/persistence/rdb.rs
Closes the remaining untrusted-length gaps in the native RDB decode path (
src/persistence/rdb.rs), same class as #266.Gaps fixed (present identically in
read_entryandread_entry_zero_copy— literal duplicates):group_count/pel_count/consumer_count/pending_countin theTYPE_STREAMbranch had novalidate_countguard. All sibling counts (hash/list/set/zset/entry/field) were already guarded.Min-bytes-per-item traced from
write_entry's serialization layout (groups 28, PEL 36, consumers 16, pending 16) so no valid file is ever rejected — proven by a fully-populated round-trip control test.Tests: 4 DoS-rejection tests (crafted
u32::MAXcounts with zero backing bytes, assertedErrfrom BOTH decode paths) + 1 overcounting-safety control. 18/18 passing; clippy-D warningsclean on both feature sets; fmt clean.Note: today these counts grow maps incrementally (no eager
with_capacity), so this is defense-in-depth + pattern consistency rather than a live OOM — but it removes a latent trap for future refactors.Follow-up (not this PR):
rdb.rsis over the 1500-line guideline (~2038 lines) and needs a module split.Summary by CodeRabbit