Repository navigation
Library Scan: duplicate groups ignore size, so different releases of one album hide each other from RED/OPS #4273
Replies: 2 comments
Agent BriefCategory: enhancement Current behavior: Desired behavior: Key interfaces:
Acceptance criteria:
Out of scope:
Domain:
Logs:
Why a human: Intent conflict: |
|
Point 2 from #4210 (comment) might assist here |
Uh oh!
There was an error while loading. Please reload this page.
Description
Library Scan deduplication groups torrents by their parsed release name, without comparing size. Different releases of the same album end up in one "duplicate group", and qui searches only the group's representative. The other members never get a RED/OPS lookup of their own, and qui records them as searched. Examples of such pairs: 24-bit and 16-bit, FLAC and V0/320, or two editions whose names parse alike.
This affects Gazelle matching in two ways:
propagateDuplicateSearchHistorythen stamps the hidden member's search history when the representative's lookup completes, so it also waits out the cooldown.deduplicateSourceTorrentsininternal/services/crossseed/library_scan.go(develop) callsmatchVerdict(..., matchPolicy{SkipSizeCheck: true}), with the comment "Dedup has no sizes in its verdict". The docs and #3513 describe a group as "duplicate copies" of the same content, which a size-blind name match does not ensure.Concrete case:
Acid Kalle - VIBES (2026) [WEB FLAC]: OPS, 24-bit, 605,001,805 bytes, categorymusic-seed.Acid Kalle - VIBES [2026] [WEB FLAC]: OPS, 16-bit downconversion of the same album, 323,815,054 bytes, categorycaesura, different files.The overnight Library Scan logged
[CROSSSEED-DEDUP] ... originalCount 3017, deduplicatedCount 1926. The run's results contain the 24-bit torrent and not the 16-bit one, and the 16-bit has no search history row. Both torrents exist on RED. I fetched RED's .torrent of the 16-bit and verified all of its pieces against the local files. RED's torrent hashes exactly to the local info dict with the source flag swapped, so stage 1 alone would have matched it if the 16-bit torrent had been searched.Scale: after the scan, 109 local OPS/RED torrents had a byte-identical copy on the other site that was not seeded. That is my own tooling's count, with every piece verified on 4 of them so far.
These counts come from comparing names, so they show the pattern but don't prove that each torrent was grouped.
Expected: a duplicate group should contain only copies of the same content. For example, compare total size (or the file list, which the second pass already fetches for contested groups) in addition to the name verdict. Different releases of one album should each get their own lookup and their own history row, and a group's sites should only come from true copies.
Steps to reproduce
Artist - Album (2026) [WEB FLAC]andArtist - Album [2026] [WEB FLAC]). The 16-bit also exists on RED.cross_seed_search_historyrow. The RED copy of the 16-bit is never found.Variant for effect 2: also seed a V0 of the same album from RED. Then the FLAC is never looked up on RED at all.
Environment
Log link
No response
Relevant logs
Checklist
All reactions