feat(mft): for_each_record — streaming fixed-up targeted record reads#577
Merged
Conversation
Implements docs/dev/architecture/UFFS_MFT_RECORD_READ_ASK.md: the targeted per-candidate MFT re-read (resolve_frs_to_lcn) already reads every record's full bytes and discarded them. Expose those bytes. - New public uffs_mft::for_each_record(volume, frs_list, visit): reads each distinct FRS in ascending order, applies the NTFS Update-Sequence-Array fixup, and hands the bytes to the callback as a borrow from one reused scratch buffer — bounded memory, no per-record allocation, per-record failures fold to None without aborting the pass. Works on live volumes and VSS snapshot devices (same VolumeHandle contract as resolve_frs_to_lcn). - resolve_frs_to_lcn now delegates to for_each_record (identical output contract). Side effect: its parse now runs on fixup-verified bytes — previously it parsed raw bytes, so a runlist straddling a sector boundary could read the USA sentinel instead of data. - first_data_lcn (runlist → first real $DATA LCN) is now public, so one for_each_record pass yields LCN ordering and record content without a second call or a consumer-side runlist parser. Tests: USA fixup round-trip and torn-write/bad-magic rejection, a resident $DATA value straddling the sector-1 boundary proving fixup is load-bearing, sort/dedup ordering, and an ignored live-volume parity test (for_each_record + first_data_lcn == resolve_frs_to_lcn).
githubrobbi
marked this pull request as draft
July 22, 2026 23:55
…ree LCN path Downstream review of for_each_record, three items: - first_data_lcn no longer allocates: AttributeRef::data_runs_iter() (new, public) decodes mapping pairs lazily via the new DataRunIter, and first_data_lcn stops at the first non-sparse run. data_runs() / parse_data_runs / extract_data_runs_from_attribute are now collect() wrappers over the lazy decoders — identical results, frozen by the existing runlist tests. - The callback now receives RecordOutcome instead of Option<&[u8]>, separating NotInMft (benign absence) / Io (transient read failure) / Corrupt (record exists, failed USA fixup — itself a forensic signal) from verified Bytes. RecordOutcome::bytes() collapses to the old Option semantics for callers without forensic interest; resolve_frs_to_lcn output is unchanged. Designed in now, while the API is unreleased, so the signature never needs reopening. - Doc precision: the bounded-memory promise is two reused record-sized buffers (reader-internal aligned I/O + fixup scratch), not one.
…mask docs - New AttributeRef::is_unnamed(): const, allocation-free header check for the primary-stream test. name().is_none() decoded the full UTF-16 name into a Vec just to discard it; first_data_lcn now uses the predicate, making the whole per-candidate parse allocation-free (the earlier round fixed only the runlist side). Regression test: a named $DATA stream (ADS) must be skipped by first_data_lcn. - Docs: for_each_record / resolve_frs_to_lcn now state that frs_list holds 48-bit MFT record numbers — mask the sequence off a full file reference (file_reference & 0x0000_FFFF_FFFF_FFFF) or the read silently addresses the wrong record. Module-doc buffer wording aligned with the two-buffer reality.
githubrobbi
marked this pull request as ready for review
July 23, 2026 00:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The targeted per-candidate MFT re-read (
resolve_frs_to_lcn) already reads every requested record's full bytes off the volume — then parses out a single LCN and discards them. Downstream consumers need those bytes (resident$DATAcontent, reparse targets, forensic attributes) and were paying for the read anyway. This exposes them, streaming and bounded.API
uffs_mft::for_each_record(volume, frs_list, visit)— reads each distinct FRS in ascending order, applies the NTFS Update-Sequence-Array fixup, and hands the fixed-up record to the callback as a borrow from one reused scratch buffer: no per-record allocation, retention impossible by construction. Per-record failures (outside$MFT, transient I/O, torn write) fold toNonewithout aborting the pass;Erronly when$MFT's extents can't be determined at all. Works on live volumes and VSS snapshot devices — the sameVolumeHandlecontract asresolve_frs_to_lcn.MftRecordReaderstayspub(crate).resolve_frs_to_lcnnow delegates to it (identical output contract). Side effect: the LCN parse now runs on fixup-verified bytes — previously it parsed raw record bytes, so a runlist straddling a sector boundary (offset 510–511) could read the USA sentinel instead of data.first_data_lcnis now public — callers derive the LCN from the callback bytes with the exact parserresolve_frs_to_lcnuses, so one pass yields LCN ordering and record contents with no second call and no duplicated runlist parsing.Hot-path impact: none
Purely additive. The bulk-ingest readers and the search path never touch this module; the only in-repo caller of
resolve_frs_to_lcnis the opt-inresolve_lcn_orderpost-processing pass in the daemon, which is unchanged in behavior and gains only a ~1 KiB memcpy + fixup per candidate — noise against the disk-bound read it wraps. No index schema, wire, or memory-footprint changes.Tests
$DATAvalue straddling the sector-1 boundary parses to real content only after fixup (raw bytes hold the sentinel).--ignored, elevated Windows):for_each_record+first_data_lcnmust reproduceresolve_frs_to_lcnexactly, callbacks ascending + deduplicated, out-of-MFT FRS folds toNone.Gates: full workspace suite (2498), lint-prod, lint-tests, xwin Windows clippy, rustdoc (
--document-private-items --locked), doc-tests — all green; full pre-push bundle passed on push.