Skip to content

v0.15.0

Choose a tag to compare

@github-actions github-actions released this 28 Oct 22:08
· 354 commits to main since this release
8159766

Upgrade Notes

  • No operator action required. This release ships the ARCH-030 missing-track
    recovery and Import Review redesign (a new user-facing feature family and new
    recovery/import endpoints), which is why the minor version moves to 0.15.0.
    On first boot the backend creates a new crash-safe SQLite store,
    recovery_carriers (WAL), under WEB_MANAGER_DATA_DIR; it self-initializes
    and manages its own schema, and nothing rewrites your existing
    /web-manager-data state, the Beets config.yaml, beetsplug/, or the music
    library DB. The webmanager plugin version is unchanged (PLUGIN_VERSION 1.19.0,
    PROTOCOL_VERSION 1.0), so Beets does not need a plugin-driven restart.

Security

  • A contradicting EMBEDDED recording ID now vetoes an otherwise-corroborated NO_RESULT slot, closing the F1-RECID auto-insert of an ambiguous recording (ADR-0007 req-5, music-identity). The production tag reader acoustid_service._read_file_media_tags read mb_trackid only in the beets.mediafile branch, which embedded Beets 2.14 never populates (MediaFile is now the standalone mediafile package, not beets.mediafile), so the PRODUCTION mutagen/easy=True branch never read musicbrainz_trackid. embedded_identity_for_file therefore dropped the embedded recording ID, the slot proof rows (matching_service._missing_track_recording_proof) never set recording_id, and the existing recording_id_conflict in matching/track_alignment could not fire on the slot path — so a NO_RESULT file whose embedded RG+artist+title all corroborate the target but whose embedded recording ID is a DIFFERENT recording (e.g. a deluxe/remaster take of a same-titled intro/outro) auto-inserted, exactly the ambiguity req-5 sends to review. The mutagen branch now reads musicbrainz_trackid (and discnumber), embedded_identity_for_file carries mb_recording_id, and the proof threads it onto each local row as recording_id so the EXISTING conflict logic fires and is_deterministic_metadata_slot's not self.conflicts blocks auto-accept. The conflict logic itself is unchanged — only the dropped signal is fed to it. A matching or absent embedded recording ID still auto-accepts. The prior test injected recording_id straight onto the proof row (never exercising the producer), which is why the gap shipped green; the new tests drive the REAL mutagen producer end to end. Tests: tests/test_arch030_recid_position_veto.py.
  • The AcoustID release-id shortcut can no longer surface a candidate UNRELATED to the carrier's target Release Group as the primary suggestion (F7ai, req-4/req-6, music-identity). In ai_service._ai_suggest_folder_internal the Step-1 relatedness veto only SORTS the candidate pool, but the AcoustID shortcut (2+ fingerprinted tracks → skip AI) looks its winner up by release id, so a foreign fingerprint-matched release on an unrelated Release Group (a cross-RG fingerprint collision / injected fingerprint) could still win the shortcut and become the PRIMARY action, violating "an unrelated candidate must never be the primary action." When the folder is bound to a recovery carrier (_target_rg known), the shortcut now refuses an _candidate_is_unrelated winner and falls through to the AI/fallback path, where the vetoed candidate is already sunk below related ones — the carrier/target RG stays the identity constraint. A related winner still takes the shortcut unchanged; with no carrier RG the veto remains a strict no-op. Tests: tests/test_arch030_f7ai_unrelated_shortcut_veto.py.
  • Recovery staging copies files symlink-safe, closing the ARCH-030 predictable-path symlink escape (SEC-007/#325, integrations). backend/slskd.stage_selected_audio_files — the shared staging sink reached by every recovery caller (import_service._stage_selected_audio_files, acquisition_service) — previously created each per-file destination with dest.parent.mkdir + shutil.copy2, both of which FOLLOW symlinks, while the post-mkdir containment re-check validated only the top-level stage dir, not each per-file dest or the CD NN disc subdirs. Because ARCH-030 made the staged folder name PREDICTABLE (rec-<16hex>) and added reuse-overwrite, a symlink pre-planted at a dest file or a CD NN subdir — by a local writer to the app-private staging dir, or via a race between recovery retries — was followed by copy2, letting downloaded bytes overwrite an arbitrary file the app user can write (config, Beets DB, media). The copy loop now mirrors the SEC-007 barrier discipline from composite_workflows: every directory below the validated stage folder is opened or created one component at a time with O_DIRECTORY|O_NOFOLLOW (a component swapped for a symlink is refused, never followed), and each destination is written to an O_CREAT|O_EXCL|O_NOFOLLOW temp then atomically renamed into place — the rename replaces a dest planted on a deterministic reuse (symlink or file) WITHOUT following it. Staging fails closed on hosts without os.supports_dir_fd/O_NOFOLLOW (no symlink-following path fallback), consistent with the rest of the barrier. The stale comment that claimed the directory re-check "closes the predictable-path/symlink TOCTOU" is corrected — it validates only the top-level dir; the per-file fd-relative copy is what closes it. No new credentials, env vars, or logged secrets. Tests: tests/test_sec_arch030_slskd_nofollow.py (POSIX; symlink pre-planted at a dest file and at a CD NN disc subdir is refused/not-followed and its target is untouched), existing tests/test_slskd_helpers.py staging coverage unchanged.
  • Recovery staging now descends the intermediate _beets_missing_import component fd-relative instead of following it (SEC-007/#325 Finding F1/C5, integrations). backend/slskd._staged_copy_nofollow opened the FULL validated stage path (<downloads>/_beets_missing_import/<prefix>-album) in a single os.open(..., O_NOFOLLOW); O_NOFOLLOW guards only a path's FINAL component, so the intermediate _beets_missing_import directory was still followed if it were a symlink — and the stage.resolve() containment check in stage_selected_audio_files is a separate syscall (TOCTOU). Escaping containment required winning a tight race on the intermediate component with a co-located local writer (outside the single-container threat model), and resolve() still blocked an outside target — hence non-blocking — but it did not match the fd discipline already used for the deeper components. The copy now opens the trusted downloads root once (without O_NOFOLLOW, as the configured mount, mirroring _unlink_proven_under) and descends EVERY component below it — the intermediate _beets_missing_import staging dir, the <prefix>-album folder and each CD NN disc subdir — one at a time with O_DIRECTORY|O_NOFOLLOW, failing closed (RuntimeError) on any symlinked or non-directory component, and still fails closed on hosts without os.supports_dir_fd/O_NOFOLLOW. The Finding-A per-file O_CREAT|O_EXCL|O_NOFOLLOW temp + atomic rename plant is unchanged. No credentials, env vars, or logged secrets. Tests: tests/test_sec_arch030_slskd_nofollow.py (POSIX; a symlinked intermediate _beets_missing_import pointing at an in-root decoy — so resolve() passes and only the fd-walk can refuse — is refused and no bytes are written through it; the existing dest-file and CD NN cases still pass).
  • Orphan staging cleanup fails closed when the recovery staging root itself is a symlink (SEC-007/#325 Finding B, ARCH-030 Step 4). recovery_service._staging_root resolved <downloads>/_beets_missing_import with os.path.realpath, which silently FOLLOWS a symlinked root: if _beets_missing_import were a symlink (attacker pre-plant or a benign relocation symlink) pointing at the whole <downloads> root, root became that target and the per-child realpath/commonpath containment check in reconcile_orphan_staging_folders then accepted the target's aged, unowned top-level entries — so the sweep could delete_file not-yet-imported completed downloads at the downloads root, contradicting the function's own guarantee never to delete real media outside the staging root. (The live music library stayed protected by delete_file's own MUSIC_ROOT/protected-data refusal.) _staging_root now lstats that path (os.path.islink, which never follows the final component) and returns "" — no cleanup — when it is a symlink, so root is only ever the genuine staging directory and the existing per-child containment check can only ever match entries truly inside it. The composite_workflows.delete_file barrier is unchanged. Tests: tests/test_mtr_step4_gate_and_orphans.py (SymlinkedStagingRootTests).
  • Orphan staging cleanup fails closed on carrier-store uncertainty instead of treating a read error as "unowned" (SEC-007/#325 Finding F8, ARCH-030 Step 4). recovery_service.reconcile_orphan_staging_folders decided a staging folder was deletable from two carrier-store reads — the per-folder recovery_carrier.load_for_folder ownership lookup and the _live_carrier_prefixes() live-carrier map. Both swallowed errors into a negative result (bound=None, empty prefix map), so a transient carrier-store read failure made an aged folder that actually belongs to a LIVE carrier look unowned, and the sweep could delete_file it. (The deletion stayed contained to the staging root and recoverable, but it was still an unintended removal of a live recovery's staging folder.) A deletion now requires that BOTH ownership reads positively SUCCEEDED and showed no owner: _live_carrier_prefixes() no longer swallows a store-read error (it raises, so an empty return is a trustworthy "no live carriers" rather than an error-empty), the caller records a per-run prefix-read failure, and the per-folder lookup records its own failure; if either read failed, ownership is UNKNOWN and the folder is skipped (never deleted), logging only the exception type and folder name. Adoption and the normal unowned-and-aged cleanup are unchanged. No credentials, env vars, or secrets are involved. Tests: tests/test_mtr_step4_gate_and_orphans.py (test_per_folder_lookup_failure_fails_closed_no_delete, test_prefix_store_read_failure_fails_closed_no_delete).
  • Orphan staging cleanup now protects a queued Review path that is an ancestor or descendant of a scanned folder, not just an exact match (ARCH-030 reconcile-overlap, library-transaction). recovery_service.reconcile_orphan_staging_folders scans TOP-LEVEL staging folders but checked the Review queue with _pending_review_has_path, which is exact-path membership only. A queued Review row at a DESCENDANT of a scanned orphan (e.g. a row at <prefix>-album/CD 02 while the sweep deletes <prefix>-album) was unprotected, so an aged unowned parent folder could be deleted out from under a not-yet-imported download. The sweep and import_service._delete_staged_import_folder now use a new pending_review_store._pending_review_has_path_or_overlap, which keeps the folder when any queued Review path equals, is a descendant of, or is an ancestor of it — boundary-aware (keys are resolved/normalised, so .../album never matches .../album-2) and still fail-closed (unknown=True) on an unreadable queue. Tests: tests/test_arch030_merge_completion.py (PendingReviewOverlapTests).
  • Recovery staging-root resolution fails closed when the /proc/self/fd read-back is unavailable, instead of re-resolving by name (SEC-007/#325, ARCH-030, library-transaction). recovery_service._staging_root resolves the staging dir through a single O_NOFOLLOW fd-open and reads its canonical path back from /proc/self/fd/<fd>; on an fd-capable host where that read-back failed it fell back to os.path.realpath(os.path.join(base, name)), which drops the pinned fd and re-resolves the whole path by name — reopening the TOCTOU window the fd-open had just closed (an intermediate component swapped to a symlink between the open and the realpath would be followed). It now returns "" (no cleanup) on a read-back failure, keeping root only ever the genuine fd-pinned staging directory. Tests: tests/test_arch030_merge_completion.py (StagingRootReadbackFailClosedTests).
  • Deferred-file separation fails CLOSED, so a leftover unverified file can neither bypass the slot gate nor be deleted with the retained acquisition (ARCH-030 F8, integrations). acquisition_service._separate_deferred_slot_files swallowed per-file move failures (WARN only) and returned a truthy slot folder if ANY file moved, so _import_downloaded_album proceeded UNCONDITIONALLY to _validate_import_source_audio + the inline Beets reimport on the ORIGINAL folder. A deferred NO_RESULT/UNAVAILABLE file that failed to move out (collision / symlink refusal / I/O) therefore (a) stayed in import_dir and rode into Beets autotag, bypassing the DETERMINISTIC_METADATA_SLOT gate ("no track bypasses identity verification", req-5), and (b) was destroyed if a later import-validation raise triggered _delete_staged_import_folder on the whole folder — the opposite of B3 "retain, never delete". _separate_deferred_slot_files now also reports whether EVERY deferred file was relocated, and the caller runs the inline import only when separation is COMPLETE; on an incomplete separation it does NOT inline-import and does NOT delete, leaving the whole carrier STAGED for the metadata-slot recovery gate (the existing all-NO_RESULT staged-for-gate path is unchanged). No new credentials, env vars, or logged secrets. Tests: tests/test_arch030_slot_evidence_staging.py (unit: one unmovable deferred file → incomplete → retained in import_dir; e2e through start_album_download: a mixed batch where one deferred move raises → no inline import, no delete, carrier staged).
  • Restaging an already-staged subset no longer launders the genuine download name into the manufactured staged name (ARCH-030 F2, integrations). slskd.stage_selected_audio_files recorded src.name as each staged file's "original pre-rename download name" sidecar entry; when the SOURCE is itself an already app-staged/renamed folder (a restage of the still-missing subset via the import_service restage call site), src.name is the manufactured {track:02d} {title} staged name, so the genuine original was overwritten and Option-C's two independent corroboration sources (original-name + embedded-tags) collapsed to embedded-tags-only. The staging loop now carries the prior original forward from the source folder's sidecar (original_download_names_for_folder(source)), recording src.name only for a genuine fresh download with no prior sidecar — the same relocation discipline _separate_deferred_slot_files already uses. Tests: tests/test_arch030_slot_evidence_staging.py (restage carries the genuine original download name, not 01 Song.flac).
  • The recovery original-names sidecar is read and written fd-relative, closing an intermediate-parent symlink follow and a hardlink truncation (ARCH-030 F4/SEC-007/#325, integrations). slskd._read_text_nofollow / _write_text_nofollow opened the sidecar by FULL path with O_NOFOLLOW, which guards only a path's final component — an intermediate parent-dir symlink swap was still followed — and the write used O_TRUNC with no O_EXCL/st_nlink guard, so a planted hardlink at the sidecar could be truncated. Both helpers now open the sidecar's parent dir with O_DIRECTORY|O_NOFOLLOW and operate on the leaf fd-relative, mirroring the sibling _staged_copy_nofollow; the write goes to an O_CREAT|O_EXCL|O_NOFOLLOW temp and is atomically renamed onto the sidecar name, so no O_TRUNC ever touches the real file and a planted symlink or hardlink at the leaf is replaced, not followed or truncated. The non-POSIX plain read/write fallback is unchanged. No credentials or secrets are involved. Tests: tests/test_sec_arch030_slskd_nofollow.py (symlinked sidecar leaf and symlinked parent dir refused; a planted hardlink is not truncated).

Fixed

  • The slot gate now cross-checks disc/track against evidence INDEPENDENT of the circular staged filename, ending the trivially-satisfied position check (F1-POSITION, music-identity). matching_service._missing_track_recording_proof derived each proof row's disc/track with _audio_position_from_path(staged_path), but slskd's stager had already renamed the file TO the target slot ({track:02d} {title}), so position_match was trivially true and a same-title-different-position file satisfied it for free. The row position now comes only from independent sources — the embedded track/disc tags (carried through file_embedded_ids) and the ORIGINAL pre-rename download basename (file_original_names), never the staged name — via the new _independent_slot_position. When neither independent source carries a position the file is treated as not-corroborated (track 0 → position_match False → review), consistent with Option C. Genuine recoveries whose embedded tags or original names carry the real position still auto-accept. Tests: tests/test_arch030_recid_position_veto.py.
  • A missing-track recovery routed to manual review is now VISIBLE and actionable in Import Review (ARCH-030 F9, backend). Under Option C the common case — a fresh download with every requested track NO_RESULT and auto_import=true — stages the files and the scheduled advancer, finding the deterministic slot gate correctly declines, lands the carrier in review_required by transitioning carrier state ONLY; it never enqueues a pending-review row. GET /api/import/review-queue built its list solely from backend.pending_review_store._load_pending_reviews, and there is no carrier-list endpoint, so a review_required carrier that was never enqueued was INVISIBLE: the user could neither see nor act on their missing-track recovery (defeating req-3/req-6/req-8 for the main Option-C path). The review-queue endpoint now ALSO surfaces recovery carriers stuck in review_required that are not already represented by a pending row, building each item from the carrier's own recovery_carrier.review_identity packet — the SAME shape the existing RecoveryReviewCard renders (recovery_carrier_id, canonical mb_releasegroupid, existing_album_id, wanted/expected tracks with per-track status, matching_decision, review_reason) — deduped against the pending rows by carrier id and staged path so an already-enqueued carrier is never doubled. The read is bounded (list_carriers_by_state, capped) and surfaces recovery_review_truncated when the cap is hit rather than silently dropping carriers. The gate/auto-insert authority and the carrier state machine are unchanged; this is a read-only display surface (only review_required is shown — the other advanceable states are still progressing and the advancer re-evaluates them). No frontend change was required: the existing card already renders carrier-packet items. Tests: tests/test_arch030_recovery_review_visibility.py (drives the REAL advancer: a NO_RESULT download the gate declines lands in review_required and IS surfaced with its packet; an auto-accepted carrier is not surfaced; an already-enqueued carrier is not doubled).
  • Recovery staging preserves two GENUINE, independent evidence sources per staged file for the Option-C slot gate, ending the title/position laundering (ARCH-030 B1, integrations). slskd.stage_selected_audio_files._target_dest renames every staged file to {track:02d} {title} from the REQUESTED track, and the slot proof then read the local title/position back OFF that manufactured filename (matching_service._slskd_title_guess_from_name / _audio_position_from_path) — so the gate was comparing the target against evidence the system invented, making title+position vacuous. Two path-keyed maps are now produced for the matching proof to consume (music-identity owns the gate side): (1) file_original_names Dict[staged_abs_path -> original pre-rename download basename] — captured at rename time and persisted in a dot-prefixed .bwm_recovery_original_names.json sidecar in the staged folder (survives to the advancer), read back by slskd.original_download_names_for_folder(folder); (2) file_embedded_ids Dict[staged_abs_path -> {"title","mb_releasegroupid","mb_albumartistid","mb_artistid"}] read from the ACTUAL staged file's tags (which survive the rename) via acoustid_service.embedded_identity_for_file(path) / embedded_identity_map_for_folder(folder), with {} when the file carries no readable MB identity tag (→ the gate treats absent as "cannot corroborate" → manual). acoustid_service._read_file_media_tags is extended to expose the embedded title (mutagen branch) and mb_albumartistid/mb_artistid/mb_releasegroupid (both branches). The import rename is unchanged; only the genuine evidence now survives alongside it. No credentials or secrets. Tests: tests/test_arch030_slot_evidence_staging.py.
  • A mixed download batch (≥1 verified + ≥1 deferred) imports ONLY the verified files inline; deferred NO_RESULT/UNAVAILABLE files never ride into Beets (ARCH-030 B2, integrations). acquisition_service._import_downloaded_album returned the staged folder to the slot gate only when ZERO files verified; when a sibling verified, it fell through to the inline reimport on the WHOLE folder, so deferred files still physically in import_dir were dragged into Beets autotag and bypassed the slot gate ("no track bypasses identity verification", req-5). The validator now also returns the deferred files' staged paths, and before the inline reimport those files are physically separated into a carrier-bound …__slot_deferred sibling folder (intra-staging-root os.replace, not a copy from an untrusted source — it does not re-open the SEC-007 copy surface), carrying their original-name evidence with them; only the verified files import inline and the deferred files reach the slot-gate/manual path only. Tests: tests/test_arch030_slot_evidence_staging.py (mixed batch imports only the verified track; the NO_RESULT file never reaches the import folder).
  • Real AcoustID "could not ask" statuses are now DEFERRED, not deleted (ARCH-030 B3, integrations). acquisition_service._download_identity_deferral mapped only "failed"/"not_attempted" → defer, but production (acoustid_service._audio_identity_decision → acoustid_failure_status) emits not_configured/auth_failed/lookup_failed for the real not-configured/rejected-key/outage/throttle/missing-fpcalc cases, which fell through to "" → DELETE. All of these are now treated as UNAVAILABLE → DEFER (retain staged for manual review, never auto-import, never delete); genuine NO_RESULT still defers and a contradicting candidate still deletes. The earlier test only mocked "failed" — which the provider path never emits — which is why the bug shipped green; the new tests drive the REAL statuses through _audio_identity_decision. Tests: tests/test_arch030_slot_evidence_staging.py.
  • The download-time identity gate now DEFERS a genuine AcoustID NO_RESULT recovery file to the slot tier instead of deleting it before it can get there (ARCH-030 C1, integrations). acquisition_service._validate_wanted_download_identity_before_import set match["ok"]=False for a file whose AcoustID lookup answered empty (NO_RESULT) exactly as for wrong audio, so best_idx<0 and composite_workflows.delete_file removed it — and, when the whole download was NO_RESULT (the 112 "Forever" intro/outro anchor case), the staged folder became empty, _first_staged_folder_with_audio returned "", and the DETERMINISTIC_METADATA_SLOT tier shipped in Step 6 was UNREACHABLE in the real flow: the feature could not complete end to end for its own anchor case. The gate now classifies a non-accepted file from its AcoustID identity status: a genuine NO_RESULT (answered-empty — the file may be the correct intro/outro, and NO_RESULT is exactly what the slot gate requires) or an UNAVAILABLE lookup (not configured / rejected key / outage / rate limit / timeout / fpcalc missing — not negative evidence) is LEFT STAGED, not deleted; only a CONFLICT (AcoustID returned a candidate that contradicts the wanted recording/RG) or a format-rejected file is still deleted. When no file fingerprint-accepts inline but one or more were deferred, _import_downloaded_album does NOT run the inline Beets import on unverified audio — it leaves the carrier STAGED (an advancer state, file on disk) so the scheduled advancer's DETERMINISTIC_METADATA_SLOT gate — which remains the sole auto-insert authority and still must pass — decides accept-vs-review downstream. No gate is weakened: the fail-closed "a file we decided to delete that cannot be removed → nothing imports" guard is preserved, and the validator still raises when nothing verified AND nothing was deferred. Tests: tests/test_arch030_noresult_download_defer.py (drives the real download → validate → stage → advance path: a NO_RESULT file survives staging and the slot-tier folder resolver finds it; a CONFLICT file is still deleted).
  • Bare-release-ID missing-track recoveries no longer mint duplicate carriers/folders (ARCH-030 C9, integrations). acquisition_service.start_album_download computed the recovery dedup/resume identity key from the requested rgid, but create_carrier keys on the RESOLVED canonical rgid. A client that supplied only a release id (mb_albumid, no mb_releasegroupid) is indistinguishable by UUID shape from a release-group id, so the key used the release id while the carrier was created under the real Release Group — get_by_identity missed it and a retry minted a DUPLICATE carrier/folder. A recovery with no explicit mb_releasegroupid now resolves the release's canonical Release Group up front (helpers_mb._fetch_mb_release_group_strict) so the dedup key, the identity-match guard and create_carrier all agree; it fails open to the bare id when MusicBrainz cannot answer (worst case: a lost resume optimisation). An explicit mb_releasegroupid and every non-recovery keep the existing network-free path. Tests: tests/test_arch030_noresult_download_defer.py.
  • (C10, documented) The pre-lock get_by_identity dedup read in start_album_download runs before the guarded job's identity workflow lock, so two concurrent duplicate requests can both read "no carrier" and mint a fresh uuid; this is benign — the job workflow is keyed on the same stable identity slug and fail_fast_in_process refuses the second job, so only one _do runs and only one carrier is ever created. Marked in code with a ponytail: comment naming the residual race; moving the read under the job-engine guard is not a small, safe change and is unnecessary given the lock already serialises creation.
  • The production recovery preflight now FEEDS the Option-C slot proof the genuine evidence it already knew how to consume, so a real corroborated missing-track recovery auto-completes instead of always fail-closing to review (ARCH-030 slot-evidence wiring, music-identity). The integrations producers exposed the two independent corroborators only through folder READERS (slskd.original_download_names_for_folder(folder) for the sidecar's pre-rename download names; acoustid_service.embedded_identity_map_for_folder(folder) for each staged file's embedded MB tags), and the matching proof (_missing_track_recording_proof) already consumed file_original_names / file_embedded_ids — but _folder_release_preflight built those two maps from candidate-row fields (original_name / embedded_*) that nothing populates on the production local-scan path, so both arrived EMPTY and every recovery (including the genuine 112 "Forever" NO_RESULT anchor) hit the double-corroboration gate with no evidence → file_unverified → manual review forever. The proof call site now sources both maps from the readers, keyed to the SAME staged abs paths as proof_file_hits/audio_files, with the pinned contract field names ("title","mb_releasegroupid","mb_albumartistid","mb_artistid") produced verbatim by acoustid_service; any candidate-carried values (engine-inspection topology) only fill gaps. The gate DECISION logic is unchanged (double-corroboration, F4 version-launder, F7 RG/artist contradiction all as-is): absent evidence still stays absent → cannot corroborate → review. A fully corroborated recovery (original download name AND embedded tags both matching title+position+duration±7s+RG+artist) now reaches action_allowed=True end to end; launder/absent-tag/contradicting-RG cases still go to review. Tests: tests/test_mtr_arch030_slot_evidence_wiring.py (drives the REAL preflight against an on-disk staged folder with a real sidecar — not hand-built candidate dicts — and asserts verdict-level auto-accept vs review for all four cases).
  • An authorized (or human-confirmed) NO_RESULT missing-track recovery now completes the merge into the existing album instead of stranding a duplicate album row (ARCH-030 B4, library-transaction). The post-import AcoustID gate import_service._validate_wanted_album_items_with_acoustid refused on no_result unconditionally, so even the headline 112 NO_RESULT "Forever" anchor case — a slot the matching gate had already authorized (matching_decision.action_allowed=True, VERIFIED_SUBSET / DETERMINISTIC_METADATA_SLOT under the Option C double-corroboration policy, docs/adr/0007) — was imported by Beets and then REFUSED the merge, leaving the downloaded files as a separate, stranded album row next to the existing album. The gate now HONORS that already-set authorization for exactly the authorized wanted subset: when the folder-bound carrier is action_allowed=True (read, never recomputed, via the new consume-only _recovery_slot_authorized), an AcoustID no_result for the authorized tracks no longer blocks the merge, so the fill lands on the existing album (fs move + DB merge) on both the unattended advancer path and the human-confirmed manual-review path (where, under Option C, most recovery tracks go). Everything stronger still refuses even when authorized — a different-song mismatch, an infra not_checked failure, and any file that maps to no requested track (unverified, i.e. outside the authorized subset) — and an unauthorized/unverified recovery still goes to Review unchanged. Completion is still proven fs+db (the existing read-back on the existing album), so nothing is marked imported until the files are actually merged. Tests: tests/test_arch030_merge_completion.py (an integration test reaching the REAL merge: an authorized NO_RESULT slot merges with no stranded row, while an unauthorized one still goes to Review; plus the validator subset-boundary and _recovery_slot_authorized cases).

Changed

  • The DETERMINISTIC_METADATA_SLOT auto-insert gate is hardened to Option C hybrid double-corroboration — a NO_RESULT recovery auto-inserts only when the ORIGINAL download name AND the embedded MB tags BOTH corroborate (ARCH-030 round 2 B1/F4/F7/B5, music-identity). Two independent reviewers failed the prior slot gate; the USER chose a stricter policy. A genuine AcoustID NO_RESULT missing-track recovery now reaches action_allowed=True (via the slot / VERIFIED_SUBSET tier) ONLY when, per present track: (1) the version-PRESERVING target title equals a canonical form of BOTH the ORIGINAL pre-rename download name AND the file's EMBEDDED title tag; (2) exact disc+track position; (3) measured duration within 7.0s; (4) genuine NO_RESULT, no conflicts; (5) candidate Release Group == target RG; and (6) the file's EMBEDDED Release-Group MBID AND artist MBID are both PRESENT and equal the target. If either the original-name title or the embedded-tag set is absent or disagrees on title/RG/artist, the slot fails closed to manual review — so a fresh download lacking embedded MB tags goes to review (the intended safe behavior). F4 (security runtime-proven): backend/matching/normalize.canonical_title_identity_variants built strict identity forms from BOTH the title AND the filename stem, and the track-number strip laundered a version-bearing authoritative title (Song (Live 2019)) down to a bare name (01 Song.flac) that matched a bare target — proven: canonical_title_identity_variants('Song (Live 2019)','01 Song.flac') -> ['song live 2019','01 song','song']. The two sources are now independent (title-only when a title is present; filename stem only as a fallback), and the per-track title gate (models.is_deterministic_metadata_slot / _title_double_corroborated) reads the ORIGINAL download name and the embedded title — never the staged/renamed filename the system set to the target (the prior circular _slskd_title_guess_from_name(Path(path).name) title). B5/F7: the recovery candidate dicts now thread the canonical artist MBID (_fetch_mb_release_tracklist.release_artist_id / mb_albumartistid) so the F7 artist-contradiction check fires and the embedded-artist corroboration (evidence._row_embedded_corroborates) has a target; evidence._artist_id now also reads mb_albumartistid. _missing_track_recording_proof consumes the pinned evidence contract integrations produces, keyed by staged path: file_original_names (original basename) and file_embedded_ids ({"title","mb_releasegroupid","mb_albumartistid","mb_artistid"}). Ranking (title_similarity/_title_similarity) and the ADR-0002/0003 thresholds are unchanged; FLEUR wrong-RG stays blocked. ADR-0007 Signal 1 updated to the double-corroboration. Tests: tests/test_mtr_deterministic_slot.py, tests/test_mtr_arch030_slot_wiring.py.
  • The missing-track recovery advancer now runs automatically on the job scheduler, and its keep-in-review fallback is a compare-and-set (ARCH-030 F6/F9, backend). F6: the lifecycle advancer (recovery_service.advance_stuck_carriers / start_recovery_advance_job) previously ran ONLY when something POSTed /api/import/recovery/advance — there was no scheduler, no acquisition handoff and no frontend caller — so a carrier a download left stuck in DOWNLOADED/verifying/review_required never advanced on its own and requirement 8 ("download → evaluate → auto-import when safe, else retain target identity and report why") was not met end-to-end. A new recovery_service.start_recovery_advance_schedule() registers the SAME bounded, guarded advancer on the canonical job scheduler (jobs.every, the hook maintenance's auto-scan already uses — not an ad-hoc thread), registered once at startup from app.py; each tick starts the one advancer job, jobs.every skips a tick while the returned job still runs, and the job stays workflow-locked + dedupe_key-guarded on recovery-advance, so the sweep never overlaps itself across threads, processes or restarts and never creates a duplicate import / retry storm. The trigger ONLY re-evaluates carriers against the matching contract; the action_allowed=True gate in start_folder_import_with_id (Step 4) remains the one and only auto-insert authority, and a carrier the contract does not authorize keeps its target identity and stays in review with the contract's reason. Cadence is RECOVERY_ADVANCE_INTERVAL_SECONDS (default 120s; <= 0 disables the schedule, the manual POST endpoint still works). F9 (CAS race): the fallback that keeps a non-advanceable carrier in review used a blind advance_state(cid, "review_required") with no expected_revision, so a concurrent CLAIM (-> importing) that landed during the slow preflight was clobbered back to review; it now re-reads the carrier, only transitions one that is STILL advanceable (never dragging an already-claimed/terminal carrier back), and compare-and-sets on the re-read revision — on a CAS miss it skips rather than overwrites, leaving the next sweep to re-evaluate. No new job type, mutation sink, credentials or env secrets. Tests: tests/test_mtr_arch030_advancer_scheduling.py.
  • The recovery insert authority now consumes the DETERMINISTIC_METADATA_SLOT tier, so a genuine AcoustID NO_RESULT missing-track recovery is proven instead of stalling in review (ARCH-030 Step 6, music-identity). backend/matching_service._missing_track_recording_proof previously required a per-file confirmed AcoustID recording, so the anchor case — 112 "Forever" (RG 37cb4a72) missing #1 "Forever (Intro)" and #9 "For Us (Outro)", both fingerprinting to NO_RESULT (genuinely absent, not an outage) — never satisfied the proof and stayed in review forever even though the matching contract already authorized it (F2). The proof now accepts a NO_RESULT file that the shared contract's own alignment proves is a deterministic metadata slot (exact normalized title + exact release-track position + measured duration within the ADR-0007 7.0s tolerance + validated Release Group + matching artist), resolving the filled slot from the contract's alignment rather than inventing a second, looser definition, and it agrees with the contract's action_allowed (VERIFIED_SUBSET) — it never authorizes a FULL_RELEASE. The fingerprint gate is NOT weakened: UNAVAILABLE, CONFLICT and AMBIGUOUS are still refused (only answered-empty NO_RESULT can qualify), and a NO_RESULT file with no measured duration fails closed to review. The slot's measured duration is fpcalc-derived (surfaced from the AcoustID answer's evidence and preserved across the file-level cache), never a local media-tag read. Separately (F7, identity robustness), backend/matching/evidence.evaluate_release_group_candidate now treats a downloaded file whose OWN embedded Release-Group/artist identity CONTRADICTS the target as a conflict (file_release_group_conflict / file_artist_conflict) routed to review — so a file self-identifying as an unrelated release (e.g. FLEUR RG 2da91738) proposed into a 112 destination (RG 37cb4a72) can never return action_allowed=True; the embedded tag is used only to DETECT the contradiction and fail closed, never to reconstruct identity. Tests: tests/test_mtr_arch030_slot_wiring.py.
  • The matching contract's action_allowed is the only authorization to auto-insert a recovery, and orphaned staging folders are safely adopted or cleaned up (ARCH-030 Step 4, library-transaction). import_service.start_folder_import_with_id now treats an UNATTENDED recovery auto-insert — marked by recovery_carrier_id, which the recovery advancer sets and a human Review confirm never does — as authorized ONLY when the matching contract attached to the folder's carrier has action_allowed=True. It consumes that server-side boolean from the carrier's stored matching_decision (it never recomputes a more permissive one) and resolves the carrier from the FOLDER binding, not the client id, so a client cannot aim an authorized carrier at an arbitrary folder. On action_allowed!=True (including a missing/empty decision — fail-closed) it refuses synchronously, before any job starts or any file is touched, returns recovery_action_not_allowed (409), and keeps the carrier in manual review with the contract's reason. A VERIFIED_SUBSET authorization still routes only the carrier's wanted subset through the existing existing-album missing-track merge, never a whole-release import. Human-confirmed Review imports (no recovery_carrier_id) are unchanged — the human is the authorization. Separately, a new bounded, best-effort sweep (recovery_service.reconcile_orphan_staging_folders, folded into the advancer job) reconciles orphaned non-terminal staging folders left under _beets_missing_import by the pre-Step-3 per-attempt scheme: a folder still bound to — or whose deterministic rec-<16hex> prefix matches — a live carrier is ADOPTED (re-bound, never re-minted), and a folder owned by no carrier and old enough is CLEANED UP through the controlled staging-delete path (composite_workflows.delete_file, LT-13). Security (SEC-007/#325): every path is pre-verified fail-closed (realpath + commonpath containment inside the validated staging root, symlinked children skipped) and delete_file independently re-enforces containment (inside a staging root, never a staging root itself, never MUSIC_ROOT or protected data, no symlink component, fd-relative, lstat-rechecked) — so the live music library is never touched and no real media outside the staging root is ever deleted. Tests: tests/test_mtr_step4_gate_and_orphans.py.
  • Import Review renders a redesigned recovery card that shows which tracks are missing, keeps the carrier's canonical album as the one primary identity, and reports per-dimension confidence + honest provider status (ARCH-030 Step 5, frontend). Carrier-backed recovery items (evidence.recovery_carrier) now render a dedicated RecoveryReviewCard with a WANTED / DOWNLOADED / WHAT-NEEDS-CHECKING / NEXT information architecture and exactly ONE primary action, replacing the old identity panel + stage pair. Which tracks are missing (req 3) is shown explicitly: expected-track titles with track/disc numbers, clickable MusicBrainz recording links and the canonical Release Group link, expected-vs-present counts, and a per-track status chip (imported / verified / downloaded / pending) read straight from the backend packet — never guessed from a folder name. Expected vs suggested (req 6) are separated: the intended destination album (the carrier's canonical mb_releasegroupid + expected_tracks) is the primary identity and the single primary action's target, taken directly from the carrier and never chosen by candidate ranking; AI/alternative candidates render in a clearly-separated, non-primary "Alternative suggestions" section, so an unrelated title-collision (e.g. FLEUR "FOREVER FOREVER" when the target is 112 "Forever") can never be auto-selected or shown as the primary action. Per-dimension confidence + provider chips (req 7): the five matching_decision dimensions (identifier syntax, MusicBrainz lookup, album identity, downloaded-recording verification, import eligibility) render separately rather than as one blob; provider-status chips read AcoustID "no match in database" differently from "not configured", and the AI chip shows not-run / rejected and NEVER surfaces AI confidence when the AI provider did not actually contribute (ai_status.contributed=false, or a rejected/unauthenticated key). The single primary action follows the product direction "auto when deterministic evidence is strong; ELSE manual": when the matching contract allows auto-import (action_allowed=true) it is the automatic "Import recovered track" action; when it does not, the same single slot becomes a visibly-distinct human-review "Review & import manually" confirm (F5) that routes through the EXISTING human Review import path (page startApply → confirm dialog → runApply, carrying no recovery_carrier_id, so it is human-authorized and still fully server-side validated) — it never flips the auto-gate and never targets an alternative/title-collision candidate (the carrier's canonical Release Group stays the only confirmed target). Provenance and the destination path are collapsed into an expandable details block. frontend/src/api/types.ts widens matching_decision to the contract-v3 shape (top-level identity_proof, the five dimensions, action_allowed) and adds the recovery_carrier packet fields (tracks, provenance, matching_decision, confidence, review_reason, recovery_identity_key), an AiStatus type on the ai-suggest response, and provider_health on the health-detail response. Read-only: the frontend consumes backend fields present since Step 3 and changes no backend shape. Tests: frontend/tests/RecoveryReviewCard.test.tsx.
  • Missing-track recovery reuses one staged folder per recovery, records durable download provenance, and reports provider validity honestly (ARCH-030 Step 3, integrations). The staging layer now derives a deterministic folder prefix from the stable recovery-identity key (rec-<16 hex>), so a retry of the same recovery REUSES one _beets_missing_import/<prefix>-<album> folder (slskd path) and one <artist>/<album> - <source> missing <prefix> folder (direct/yt-dlp path) instead of minting a fresh uuid/timestamp family each attempt — the duplicate-folder storm behind the 38-item "Pending AI" queue. The prefix is threaded through acquisition_service and backend/import_service._stage_selected_audio_files (which self-derives it from a carrier bound to the source folder when not passed, so every staging call site gets the reuse behaviour), and a deterministic re-stage REPLACES the prior copy rather than accumulating name.1 siblings. Security (SEC-007/#325): the now-predictable staged name is sanitised to a bounded [0-9A-Za-z._-] token (a fixed-length hex-of-a-hash, never raw identity text) and, after mkdir, the real staged path is re-checked to be a non-symlink directory still contained under the validated downloads root — closing the predictable-path/symlink TOCTOU a random uuid name previously masked (enforced for both callers; the direct path keeps the existing _download_dest_under root-containment). The carrier source provenance is extended additively to a durable record: provider/method and the discovery query at create, plus the chosen result reference (resolved MB release + staged folder) and per-file source once the download lands (folded onto the existing carrier transitions — no extra lifecycle write). GET /api/health/detail now surfaces provider_boundary.provider_health() as a structured provider_health field (last classified outcome per provider, already redacted), kept separate from the env-presence checks booleans — so a present-but-rejected AI/Lidarr/Discogs/slskd/Plex key no longer reads as healthy. Item AI-suggest (POST /api/items/<id>/ai-suggest) now emits a structured ai_status (configured vs validly reachable, with the boundary's last outcome, which knows a 401-rejected key) and a confidence_source, and never attributes the MusicBrainz/AcoustID fallback confidence to AI when the AI provider did not actually respond — so the UI can show a provider-status chip instead of a misleading "AI low". matching_decision.ai is unchanged (its boundary default is correct by design). No new credentials or env vars; no secret/token is logged or persisted. Tests: tests/test_mtr3_integration.py, tests/test_mtr_acquisition_carrier.py, tests/test_missing_import_single_staging.py.
  • Carrier-bound recovery seeds MusicBrainz candidates from the canonical Release Group, not folder/tag/filename title text (ARCH-030 Step 2.5). When a recovery/repair pins the carrier's canonical mb_releasegroupid, the AI suggestion path (ai_service._ai_suggest_album_internal) now starts its candidate pool from that Release Group's own releases — a new rgid: binding seed on helpers_mb._mb_release_search — instead of a free release:"<title>" text search. This closes the remaining vector behind requirement 4 ("do not allow generic MusicBrainz title matching to replace a known target album"): a title-collision release on an unrelated Release Group (e.g. FLEUR "FOREVER FOREVER" vs 112 "Forever", RG 2da91738 vs 37cb4a72) is now neither seeded nor ranked primary, composing with the existing Step-1 relatedness veto (which the seed makes a no-op since no unrelated RG is in the pool). On an empty rgid search (search-index lag) the fallback stays RG-bound: musicbrainz_service._mb_release_search_by_folder_tracks gains an optional release_group_id that scopes every recording query with rgid: and stamps the resulting candidates with that RG (binding, not reconstruction), rather than a free title match that could land on another group. The canonical RG is only ever taken from the carrier — never reconstructed from the folder/job/filename. A genuine new-album import (no carrier RG) is unchanged. Tests: tests/test_recovery_rg_seeding.py.

Fixed

  • The deterministic-metadata-slot auto-accept gate now uses a STRICT, version-preserving canonical-title identity check, so a live take / remaster / alt-take / edit can no longer silently fill a bare-title slot (ARCH-030 F4, music-identity). backend/matching/models.TrackAssignment.is_deterministic_metadata_slot gated the title on title_similarity >= 0.999, but title_similarity is the variant-MAXIMIZED ranking score from track_alignment._title_similarity, which deliberately strips (Live)/remaster/alt-take/feat./parenthetical text (normalize.title_variants). So a wrong recording like "Song (Live 2019)" normalized to the bare target "Song" and scored 1.0 >= 0.999; combined with exact position + duration within the 7.0s tolerance + AcoustID NO_RESULT it reached can_auto_accept(VERIFIED_SUBSET) -> action_allowed=True -> an UNATTENDED import of the wrong audio, looser than docs/adr/0007 guarantees. The slot precondition now compares the version-PRESERVING canonical target title against the local title's version-preserving canonical forms (new normalize.canonical_title_identity_variants, which keeps every version/recording marker and removes only non-identity cruft — a trailing source-ID hash and an optional leading track-number prefix) and requires an exact match; a version/recording distinction that is present on the file but absent on the target now fails closed to review. _title_similarity/title_similarity are unchanged for ordinary candidate ranking. The genuine exact-title intro/outro recovery (112 "Forever (Intro)" / "For Us (Outro)") still slots. Tests: tests/test_mtr_deterministic_slot.py (VersionStrippedTitleCannotAutoPassTests).
  • The F7 embedded-contradiction guard is now effective on the recovery preflight path, not just the contract (ARCH-030 F5/C2, music-identity). The preflight candidate rows built in backend/matching_service._folder_release_preflight (_add_candidate) and _missing_track_recording_proof dropped each file's embedded Release-Group/artist identity, so the F7 guard in backend/matching/evidence.evaluate_release_group_candidate (which reads embedded_release_group_id/embedded_artist_id on each local track) never fired on the real advancer path and a mistagged file self-identifying as a different Release Group could reach the slot's not conflicts precondition unseen. Engine/tag-supplied embedded RG/artist IDs are now threaded into both row constructions, so an inspected staged file whose embedded RG contradicts the carrier target is a file_release_group_conflict and the preflight refuses to auto-accept (identity_conflict). Absence of embedded identity (the common fingerprint-only fresh-download recovery case, where no tag is read) stays neutral — it is never a positive and cannot by itself unlock auto-accept; that path is held by the now-strict F4 title gate + exact position + duration + genuine NO_RESULT + fingerprint-conflict detection. Tests: tests/test_mtr_arch030_slot_wiring.py (EmbeddedContradictionBlocksPreflightSlot).
  • Carrier-bound MusicBrainz folder-track search no longer fabricates target-Release-Group membership for foreign releases (ARCH-030 C6, music-identity). backend/musicbrainz_service._mb_release_search_by_folder_tracks scoped each recording query with rgid:<RG> and then stamped mb_releasegroupid=<RG> on EVERY release of every matched recording. The rgid: filter scopes which RECORDINGS match, but a matched recording's releases array lists every release it appears on — including releases in OTHER Release Groups (compilations, alternate editions) — so foreign releases were falsely labeled as the target RG, directly producing the "incorrect album suggestions" the recovery redesign exists to fix. Each release is now kept only when its OWN release-group id equals the target RG; a release whose release-group id is absent/unverifiable is dropped (fail closed), never fabricated into the target RG. Unscoped (no carrier RG) generic matching is unchanged. Tests: tests/test_recovery_rg_seeding.py (test_foreign_compilation_release_is_not_surfaced_as_target_rg, test_release_without_verifiable_rg_is_dropped_when_scoped).