refactor(parsers): stop computing the identity tags two normalizers discard - #651
Merged
Conversation
…iscard #650 and #654 left one answer per identity fact in the output, both by discarding what the parsers wrote: `normalize_identity_tags` removes an ungrouped `FileType`/`FileTypeExtension`/`MIMEType`, and `drop_redundant_file_size` removes an ungrouped `FileSize`. Neither touched the source. 45 inserts across 40 parsers therefore still computed values that could not reach a caller, and the parsers still read as though `FileSize` and `MIMEType` were theirs to report -- the trap that produced the duplicates in the first place. Removes them. `FileType` deliberately stays: `normalize_identity_tags` promotes it into an absent or `Unknown` `File:FileType`, and that fallback is the only reason SQLite, EVTX, Registry Hive, HDF5, X.509 and ELF are named at all -- ExifTool's tables have no row for any of them, and it refuses to open four outright. Both normalizers stay too, as the backstop for anything that reintroduces one. fits.rs is the one behavioural loose end. It wrote `File:FileType`, `File:FileTypeExtension` and `File:MIMEType` directly -- hardcoded literals, not table lookups -- which put it out of both normalizers' reach, since they only ever removed the ungrouped spellings. `merge` gives format metadata precedence over the `File:` group, so it was the last place a parser could still outrank ExifTool's own tables, silently, had the two ever drifted. The generated tables carry FITS in full (the `SIMPLE = {20}T` magic number, the `fits` extension row, `("FITS", "image/fits")`) and answer exactly as those literals did. Verified (instruments named): - Full-JSON diff of both binaries over 207 combined-samples, this branch against main @9270f94a: no key lost, none gained, no value changed. The cleanup is provably invisible, fits.rs included -- which is the evidence that removing its literals cost nothing. - `scripts/compare_file.py` over the same corpus: WRONG 134 -> 134, MISSING 1726 -> 1726, graded output byte-identical. - `cargo test --workspace`: 4840 passed, 0 failed. tests/integration.rs 574 -> 575, the new guard below; no test removed. - clippy and fmt clean; no new warning in any changed file. Guarded by `parsers_do_not_answer_identity_tags_the_file_group_owns`, which runs eight formats through the real `read_metadata` path and asserts a parser answers none of the three, while `File:FileType` still reads the expected value and `File:FileSize` keeps ExifTool's formatting rather than the raw byte count the parsers used to publish beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
swackhamer
force-pushed
the
claude/dazzling-dijkstra-438a4a
branch
from
August 10, 2026 05:03
dcf7286 to
532cb3e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope changed while this was open. It opened as a fix for
oxidex -jprinting two contradictoryFileTypelines. #650 then fixed the identity half at runtime and #654 theFileSizehalf, both by discarding what the parsers wrote. The user-visible bug is gone frommain; what is left is the source-level half neither touched, plus one behavioural loose end infits.rs.What is left
normalize_identity_tagsremoves an ungroupedFileType/FileTypeExtension/MIMEType;drop_redundant_file_sizeremoves an ungroupedFileSize. Neither changed a parser. So 45 inserts across 40 parsers still compute values that cannot reach a caller, and the parsers still read as thoughFileSizeandMIMETypewere theirs to report — the trap that produced the duplicates to begin with.This removes them, and keeps both normalizers as the backstop for anything that reintroduces one.
FileTypedeliberately stays.normalize_identity_tagspromotes it into an absent orUnknownFile:FileType, and that fallback is the only reason these are named at all — ExifTool's tables have no row for any, and it refuses to open four outright:File:FileType.sqliteSQLiteError: Unknown file type.evtxWindows Event LogError: Unknown file type.regRegistry HiveError: Unknown file type.hdf5HDF5Error: Unknown file typeEXE.elfELFELF executableDeleting the parser's
FileTypewould have dropped all five toUnknown. That is what an earlier revision of this PR did, and it is why this one does not.fits.rs— the one behavioural loose endFITSParserwroteFile:FileType,File:FileTypeExtensionandFile:MIMETypedirectly, as hardcoded literals rather than table lookups. Being already in theFile:group put it out of both normalizers' reach — they only ever removed the ungrouped spellings — andmergegives format metadata precedence over theFile:group. It was the last place a parser could silently outrank ExifTool's own tables had the two ever drifted.The generated tables carry FITS in full: the
SIMPLE = {20}Tmagic number, thefitsextension row, and("FITS", "image/fits"). Removing the literals changes nothing onFITS.fits, which is the evidence that they were redundant.Verification
Instruments named, per AGENTS.md.
combined-samples, this branch againstmain@9270f94a: no key lost, none gained, no value changed (excludingFile:FileAccessDate, which moves because reading updates atime). The cleanup is provably invisible,fits.rsincluded.scripts/compare_file.pyover the same corpus: WRONG 134 → 134, MISSING 1726 → 1726, graded output byte-identical.cargo test --workspace: 4840 passed, 0 failed.tests/integration.rs574 → 575 — the new guard. No test removed.Guard
parsers_do_not_answer_identity_tags_the_file_group_ownsruns eight formats through the realread_metadatapath and asserts a parser answers none of the three, whileFile:FileTypestill reads the expected value andFile:FileSizekeeps ExifTool's formatting rather than the raw byte count the parsers used to publish beside it.Per-parser tests that asserted the removed inserts now assert their absence, in the style
tests/forensic/lnk_tests.rsalready established.Not in scope
FileTypeinserts, for the reason above. Moving those six names intocrate::filetypewould let the fallback go, but that is a coverage change, not a cleanup.Radiance.hdrreaching the OBJ parser at all:detection/text.rsmatches"v "anywhere in the first 100 bytes. Harmless now that the parser'sFileTypeis outranked by the tables, but the routing is still wrong.🤖 Generated with Claude Code