Match scanned files when the on-disk name drops "The" or author credentials

[dny238]

Problem

The per-audiobook filesystem scan (ScanFileDiscovery.FindMatchingAudioFiles) links a file to the book only when the filename contains the full library title or the path contains the full author string (case-insensitive substring). That misses very common real-world naming:

• The file/folder dropped a leading article — Language of Emotions.m4b for the book The Language of Emotions.
• The author folder dropped post-nominal credentials or an initial — Karla McLaren\ for author M.Ed. Karla McLaren, or John M. Gottman\ vs library author John Gottman **PhD**.
• The title carries a subtitle the folder omits — Developing Mind\ for The Developing Mind, Third Edition.

In these cases the audio file sits on disk but the scan matches nothing, so the book stays fileless and Scan Folder appears to do nothing even though the file is right there.

Fix

Adds tolerant, token-based matching on top of the existing exact checks — purely additive, so any file that matched before still matches:

• Titles compared with leading articles (the/a/an) and punctuation/subtitles normalized away.
• Authors compared with honorifics/credentials (PhD, MD, M.Ed., CFP, Jr, …) and single-letter initials dropped.
• Order-independent token-subset comparison (either side may be the fuller string), which handles subtitles and middle initials.

Matching is scoped to the audiobook's own scan folder (scanRoot = audiobook.BasePath), so the extra leniency can't pull in unrelated books.

Tests

New ScanFileDiscoveryMatchTests covering: exact-title baseline (unchanged), dropped leading "The", author-with-credentials, author-with-middle-initial, title-with-subtitle vs base-title folder, and a negative case (unrelated file must not match).

• dotnet build clean (0 warnings)
• New tests 6/6; broader Scanning/Library suites 68/68 pass

🤖 Generated with Claude Code

[m4bard]

Worth flagging up front: this file is rewritten on #717. Matches is gone there, replaced by FileNameMatchesExpectedTitle(...) && HasAuthorContext(...) — title required, author demoted to corroboration — and the whole-directory grouping is gone too. I had a PR against this same method (#766) and just closed it as superseded; this one is in the same position. That's most of why I'm commenting rather than suggesting changes.

I built your branch and ran it against its own base (4555ad21, which is also current canary), so the only variable is this diff. One audiobook added, BasePath cleared so the scan root falls back to the library root, scan, then every linked file mapped back to its true owner. Library is ten public-domain books laid out {Author}/{Title}.

scanned	canary 4555ad21	this branch
Ghost Stories of an Antiquary — M. R. James	1 file (its own)	3 files — also Henry James' The Turn of the Screw and James M. Barrie' Peter Pan in Kensington Gardens
Faust I — Goethe	2 files (1 foreign: Faust I + II)	3 files — also Faust

The first row is a new misattribution, and it also moves BasePath from /audiobooks/M. R. James/Ghost Stories of an Antiquary to /audiobooks/ — the common parent of files that no longer belong together. The second row is partly pre-existing: canary already claims Faust I + II by substring, and this branch adds Faust on top.

The mechanism for the first is NormalizeAuthor dropping single-letter tokens, so M. R. James collapses to {james}, which is then subset-matched against the whole path via authorTokens.IsSubsetOf(TokenSet(NormalizeText(file))). Any file under any path containing "james" qualifies. That inherits the problem in filePathMatchesAuthor — it says "somewhere on this author's shelf", not "this book" — which the grouping above then amplifies, since one matching file adopts its whole directory. The second row is IsSubsetEitherWay accepting a subset in either direction, so an omnibus and its volumes claim each other.

Then I built the two shapes your description is about, to check what the change buys:

shape	canary	this branch	#766 (title-required)
author folder is a variant — James M. Barrie vs recorded J.M. Barrie — filename keeps the title	links	links	links
leading article dropped from folder and filename	links	links	0 files — misses

Both already work on canary, so on these two shapes the tolerant matching isn't adding recall. But the second only works on canary because of the author arm: the title arm needs the full recorded title as a substring and Turn of the Screw doesn't contain The Turn of the Screw, so the author arm is what carries it. #766 removes exactly that arm and drops to zero — which is your bug report, reproduced against my PR rather than yours.

That's the part I'd flag as worth keeping. Your NormalizeTitle handles that case directly: article-stripped, The Turn of the Screw and Turn of the Screw normalize to the same token set, so IsSubsetEitherWay matches on the title alone with no author involved. I checked that statically rather than at runtime, because on this branch filePathMatchesAuthor returns first and the tolerant title code never runs for that input.

So once #717 removes the author arm, article-dropped names lose the thing currently carrying them, and the title half of this is what replaces it. It's the author half — whole-path, initials stripped — that produces the misattributions above, and dropping it doesn't cost either shape I tested.

Repro, if it's useful — from listenarr-testdata:

./tools/validate_scan_attribution.sh --image <your-build> --asin B004FOLXEO \
    --layout author-title --only-asin B004FOLXEO,B01ATTZF38,B0C6FJ6L34

It generates the library, drives a real container, and exits non-zero if the scan claims a file belonging to another book. Happy to run it against any revision you push.

[m4bard]

Ran the harness against 0ac2326d. The misattributions I measured before are gone, and your own cases still link.

Same setup as last time. One book added, BasePath cleared so the scan walks the library root, then every linked file mapped back to its true owner.

scanned	canary 4555ad21	f9a93076	0ac2326d
Ghost Stories of an Antiquary (M. R. James)	1 file, its own	3 files	1 file, its own
Faust I (Goethe)	2 files, 1 foreign	3 files, 2 foreign	2 files, 1 foreign
Anne of Green Gables (L. M. Montgomery)	1 file, its own	2 files	1 file, its own

BasePath on the first row is back to /audiobooks/M. R. James/Ghost Stories of an Antiquary rather than collapsing to /audiobooks/. The foreign file still showing against Faust I is Faust I + II, which canary already claims by substring, so it predates this branch.

Your own shapes, same build:

shape	result
author folder is a variant (James M. Barrie vs recorded J.M. Barrie), filename keeps the title	links its own file
leading article dropped from folder and filename	links its own file

SetEquals is a better fit here than the subset test was. The article case still works, because The Turn of the Screw and Turn of the Screw normalise to the same token set. An omnibus and its volumes no longer claim each other, because theirs differ. One caveat on that first point: I confirmed it by construction rather than at runtime, since filePathMatchesAuthor still returns first for that input and the tolerant branch never runs.

You can run this yourself rather than waiting on me:

git clone https://github.com/m4bard/listenarr-testdata && cd listenarr-testdata
python3 -m venv .venv && .venv/bin/pip install -e .
./tools/vet-against.sh \
    --repo https://github.com/dny238/Listenarr.git --branch fix/scan-file-match-tolerant \
    --tool attribution \
    --asin B004FOLXEO --layout author-title \
    --only-asin B004FOLXEO,B01ATTZF38,B0C6FJ6L34

That clones your branch, builds it, generates a library holding those three books, scans, and exits non-zero if the scan claimed a file belonging to another book. It needs podman or docker, plus ffmpeg and curl.

The three ASINs are the M. R. James, Henry James and James M. Barrie books from the table, which are useful together because the names overlap. Anything in corpus/corpus.json works instead. Standalone books do need --layout author-title, because the default {author}/{series}/{title} cannot render a book with no series and skips it without saying anything.

[dny238]

Thanks for the thorough repro — the M. R. James → {james} → whole-path subset match is exactly the failure mode you measured, and you're right that dropping single-letter initials made it worse.

Pushed 0ac2326: removed the author arm entirely and tightened the title rule to strict token-set equality after article/punctuation normalization (no more either-way subset). So the tolerant path now only fires when the on-disk name equals the recorded title modulo a leading article — e.g. Language of Emotions for The Language of Emotions — and never attributes on author grounds. Added a regression test for the shared-surname cross-book case (The Turn of the Screw under Henry James/ must not match Ghost Stories of an Antiquary).

I've left the pre-existing filePathMatchesAuthor substring check alone since that's canary behavior and, as you note, #717 is already reworking it toward title-required matching. If #717 lands, the useful residue of this PR is precisely the NormalizeTitle equality — the thing that carries article-dropped names once the author arm is gone — and I'm glad to fold just that into #717's approach and close this if you'd prefer. Your call on whether it's worth keeping as a standalone or superseding.

Happy to run your validate_scan_attribution.sh against the new revision.

[therobbiedavis]

#717 replaces the old broad scan matcher with ownership/stable-identifier/book-boundary attribution and incorporates the #766/#765 author-overmatch fix. The tolerance in this PR (leading articles, credentials/initials, token-subset matching) is still additional behavior and is not superseded. Please rebase on #717 and add the tolerance inside the new book-boundary matching rules rather than reintroducing broad path matching.

[dny238]

@m4bard thanks for re-running your harness against the reworked branch — good to have it confirmed that the misattributions are gone while the real-world cases still link.

@therobbiedavis understood — I'll rebase this on #717 and add the tolerance (leading articles, credentials/initials, title token-set equality) inside the new book-boundary matching rules rather than reintroducing broad path matching. I'll wait for #717 to merge so I'm folding it into the final rules. The tolerance is title-only SetEquals now — no author fallback — so it should slot in as a normalization step rather than a widening of what counts as a match.