You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed
Recover My Uploads — rollback provenance now survives the plugin field allowlist (independent QA regression). The recovery import stamps a unique webmanager_recover_token (and source_platform) into set_fields, but the webmanager Beets plugin's validate_fields allowlist (beetsplug/webmanager/schemas.py, ALLOWED_ITEM_FIELDS) dropped both fields, so the token never persisted: the post-import token read-back returned nothing and a recovery item that was actually imported falsely reported import_failed and recorded no rollback. Both provenance flexattrs are now in the allowlist, so they round-trip through the real Beets boundary. Plugin PLUGIN_VERSION is bumped 1.18.4 → 1.18.5 (PROTOCOL_VERSION unchanged at 1.0) — the embedded Beets must be restarted/recreated for the new allowlist to take effect (the rollout script restarts Beets when the plugin version changes). The post-import token lookup (_find_recovered_item_by_token) is also made fail-closed: a single returned row carrying a different non-empty token is rejected (returns None) rather than used as a fuzzy single-row fallback, so rollback can never bind to the wrong item's file. Signed-URL scrubbing of the authorization note/evidence (_scrub_authz_text) now strips signed/secret name=value query params (X-Amz-Signature, sig, token, …) regardless of URL form, including parenthesised URLs the previous URL regex stopped short of and scheme-relative //host/… URLs it never matched. The recover-import crash-resume inference is restricted to genuine resume statuses (Running / Recovery Required / Partially Rolled Back): on a Completed transaction a missing Beets row is now refused (recover_rollback_item_missing) instead of inferred to be a crash and proceeding toward the file unlink (the content-hash guard is unchanged).
Recover My Uploads — rollback is now operator-reachable and its provenance/containment hardened (independent review). The recover_import_v1 rollback is wired into the existing POST /api/transactions/<id>/rollback route, so an authenticated operator can undo a recovery import from the UI (it dispatches to TransactionStore.rollback/rollback_recover_import, stays an explicit operator action, and is never automatic). Rollback provenance is now captured authoritatively: the import stamps a unique per-import token into the created item's set_fields and looks the item up by that exact token, so two concurrent imports of the same track can never cross-record and delete the wrong item's file (previously a fuzzy post-import (artist, title) re-query). Rollback containment now requires the recorded path to be under the currentMUSIC_ROOT only — a recorded music_root that differs (MUSIC_ROOT changed since import) is refused rather than trusted. Crash-resume is corrected so a crash between the Beets-row removal and the progress persist still resumes (the content hash still gates the unlink). The non-duplicate Beets-skip import outcome now carries the authorization record like every other outcome. own_account URL verification namespaces a /user/<name> path segment by host, so an allowlisted youtube.com/user/alice can no longer verify a different platform's source whose bare uploader_id is alice (@handle and channel_id behaviour unchanged). Signed-URL query params (X-Amz-Signature=… etc.) embedded in the authorization note/evidence are stripped before persistence.
Recover My Uploads — a newly-imported recovery item can now be rolled back, safely and audited. A recovery singleton import (_recover_import_one → beets_adapter.run_import, copy=True) previously created its transaction_engine "Import" transaction with rollback_available=False and no operations, so transactions.rollback(tx_id) raised "Rollback data has not been captured" — there was no way to undo a recovery import. The import now captures the exact, minimal undo at import time (through the audited transaction_engine store): the single added Beets item id and the single file it created under MUSIC_ROOT, with that file's sha256 content hash + size and the MUSIC_ROOT in force, recorded as the transaction's one remove_recovered_item operation with rollback_available=True (mutation family recover_import_v1). Capture is best-effort — if the copied file is not provably under MUSIC_ROOT, rollback stays unavailable rather than failing the import. Rolling back that transaction (rollback_recover_import, also dispatched by TransactionStore.rollback) removes only the recorded Beets item (through beets_adapter/the embedded engine — no raw sqlite) and the single recorded file, and only when all guards hold: the path is under MUSIC_ROOT and equals the recorded path (never a glob), with no symlinked component; the item id still maps to that path; and the file's current hash + size still match what was recorded at import (proof it is the file this import created, not something added/changed later). Any guard failure refuses the rollback and leaves the item and file in place; a hash/size mismatch (the recorded file was changed or replaced after import) is refused with recover_rollback_tampered. It never deletes via a wildcard, never touches any item or file not recorded by this transaction, never touches the staged original under DOWNLOADS_ROOT, and never asks Beets to delete files (the recorded file is deleted under the engine's own MUSIC_ROOT-containment guard, re-verified immediately before the unlink). The rollback transitions the transaction to a terminal Rolled Back status (or Recovery Required on an unexpected mid-rollback error) and logs the item id + path removed, with durable per-step progress so a crash between the DB-row removal and the file deletion resumes cleanly on retry. Authorization gate, DRM refusal, SSRF/host-allowlist guard, staged-root confinement, never-overwrite-on-import, and the identity gate are all unchanged.
Recover My Uploads — opus-in-.webm now imports. A YouTube bestaudio upload stages as a .webm (opus) container, which passes authorization, integrity, and identity but which Beets/MediaFile adds no library item from ("beets import completed without adding item to library") even though the identical audio as .opus imports. The recovery import path (_recover_import_one) now, before calling beets_adapter.run_import, detects an unimportable container by its actual ffprobe container (not just the extension; webm/matroska) and produces a lossless remux (ffmpeg -vn -map 0:a:0 -c:a copy) into the codec's native importable container (opus/vorbis → .opus/.ogg, aac/alac → .m4a); a high-quality transcode is used only as a fallback when a remux cannot yield an importable file (and is recorded as not-lossless). The converted copy is written under DOWNLOADS_ROOT/recovered (staged-root-confined, never overwriting, NUL/symlink/hard-link-safe name) and is verified before import (ffprobe-decodable, has an audio stream, nonzero size, decoded duration within a tight tolerance of the original); verification failure returns a clean import_failed with a sanitized reason and never imports. The original staged file is preserved untouched and only the converted copy is imported. The conversion (original container, converted container, lossless-remux vs transcode) is recorded on the import outcome, the provenance sidecar, and the transaction metadata. Already-importable containers (.opus, .m4a, .mp3, .flac, .ogg) are imported unchanged. DRM refusal, authorization gate, SSRF/host-allowlist guard, identity gate, and never-overwrite rule are unchanged. Optional AUDIO_FFMPEG_TIMEOUT (default 180s) bounds the conversion.
Recover My Uploads — RECOVER_OWNED_ACCOUNTS now reaches the container. The shipped Compose files (docker-compose.yml, docker-compose.full.yml) now forward RECOVER_OWNED_ACCOUNTS to the beets-web-manager service's environment: (${RECOVER_OWNED_ACCOUNTS:-}, matching the other optional operator vars), so an operator who sets the own_account allowlist in the stack .env actually has it take effect — previously it was documented and read at runtime but never plumbed into Compose, so own_account verification silently stayed inert in the shipped deployment.
Added
Recover My Uploads — authorization + provenance. Recover fetch and import now require an authorization object {basis, note} on the request. basis must be one of original_creation, own_account, licensed_permission, or open_license; a missing or invalid basis is refused with authorization basis required (...) (the probe/diagnostics routes stay open). The user's asserted claim is recorded separately from independently verified evidence on all three of the provenance sidecar, the recovery run record, and the import transaction metadata, as authorization: {asserted_basis, asserted_note, verified, verified_basis, verified_evidence, platform, recorded_at}; an asserted claim is never copied into a verified_* field. Two bases can be independently verified: own_account, when an operator-configured RECOVER_OWNED_ACCOUNTS entry matches one of the source's immutable identifiers — yt-dlp channel_id/uploader_id (exact, case-insensitive) or channel_url/uploader_url (canonical-URL / @handle / /channel/<id> / /user/<name> path-segment match), captured on probe and fetch and persisted on the sidecar; and open_license, when the source's own platform license metadata (captured from yt-dlp on probe/fetch and persisted as source_license) matches an open-license pattern (creative commons, cc-by, cc0, public domain, reuse allowed) — the verification uses the platform's license, never the user's claim. The mutable display uploader/channel name is never evidence, so a stranger's upload whose display name merely contains an allowlist token does not verify (impersonation-safe); allowlist entries shorter than 3 characters are ignored, and RECOVER_OWNED_ACCOUNTS entries are channel/profile IDs or canonical channel/profile URLs, not display names. original_creation and licensed_permission are asserted-only. The import gate uses this: a verified upload (own_account or open_license) proceeds through the existing AcoustID/MusicBrainz identity gate unchanged, while an asserted-only (unverified) upload is forced to review_required and is never auto-imported, so an operator can confirm it. The evaluated authorization record is now attached to every recovery outcome returned by the fetch and import paths — imported, skipped_existing, import_failed, and both the authorization-gate and identity-gate review_required/reject outcomes — so the asserted/verified claim is never dropped, and a retry forwards the originating run's authorization. DRM refusal, staged-root confinement, the never-overwrite-a-library-item rule, the identity gate, and the yt-dlp host-allowlist/SSRF guard are all unchanged. New config RECOVER_OWNED_ACCOUNTS (see docs/CONFIGURATION.md).
Recover My Uploads UI — authorization declaration. The Recover panel now has a required Basis for recovery selector ("I created this recording" / "Uploaded to an account I control" / "Licensed/permitted archival copy" / "Public-domain or openly licensed") and an optional note, sent as authorization: {basis, note} on Fetch and Import. Fetch and Import are blocked with an inline message until a basis is chosen, and the backend's authorization basis required error surfaces if it comes back. Each run shows its recorded authorization — the asserted-basis label, the user note, and a Verified vs Asserted (pending review) badge driven only by the backend's authorization.verified — with copy stating that an asserted (unverified) claim is routed to review, not auto-imported. No verified state is fabricated client-side.
Fix album folders whose template placeholder was never substituted. A new controlled-mutation apply complements the existing folder-placeholder scan (POST /api/library/folder-placeholders/scan): POST /api/library/folder-placeholders/apply renames album folders literally named like Hot & Wet (2003){Album MbId} (or {Track ArtistMbId}) to their clean name Hot & Wet (2003). It is a preview-first (dry_run defaults true; applying requires confirmed: true), cancellable, resumable job_engine job that goes entirely through the transaction engine — never an ad-hoc os.rename. Disk-only folders (not tracked by Beets) are renamed through the folder_cleanup_v1 engine (confined to MUSIC_ROOT, symlinks and an existing target refused, folder identity re-checked at apply, recorded for rollback). A DB-tracked folder (db_item_count>0) is not renamed by this flow — its clean destination is whatever Beets' path templates compute, which this flow cannot preview and which could merge into an existing folder via Beets suffixing — so it is held for review (reason: "tracked_needs_relocation"); relocate the album(s) directly through the album-relocation flow instead. Every row is re-verified live at apply time (source present, target still absent, still disk-only) to avoid races with the background disk reconcile, and the 52-ish target_exists collision cases are never auto-fixed — they are surfaced for manual review. POST /api/library/folder-placeholders/rollback (by job_id or explicit operations) restores every renamed folder from the audit records in one call. Triggering the apply from the existing Cleanup/placeholder-scan UI is a follow-up (frontend); the endpoints are independently callable.
Folder-placeholder apply is now safely resumable and recoverable on its error paths. The apply job's "resumable" and "rollback-able after a partial failure" guarantees now hold end to end: a row that fails after leaving a rollback record keeps its operation id in the batch so one rollback-by-job reaches the partially-applied op; each rename is written ahead to the job checkpoint before it runs, and a vanished source on resume is recovered from the transaction store, so a crash between the rename and recording its op is still rollback-able; a persistent skip (e.g. an existing target) no longer consumes the batch_limit budget, so a batched run always makes progress across resumes; and an apply with any error now returns a failed result, so a partial or total failure is no longer recorded as a successful job. resume_from_job and rollback-by-job_id now also refuse a source job that is still running (whose checkpoint would miss its later ops).
Folder-placeholder apply: resume/batch/preview-consistency correctness fixes. No data-safety regression existed; these harden recovery, budgeting, and preview-vs-apply agreement. (1) Resuming a partially-failed apply no longer duplicates the failed row's operation (a recovered op is now deduped by operation id), so rollback-by-job reports success instead of rollback_not_eligible after it has actually restored both folders. (2) A DB-tracked album whose tracks live under a disc subfolder is now detected descendant-aware at the top gate — matching the engine's own DB-reference refusal — so it is held (reason: "tracked_needs_relocation") identically in preview and apply (previously preview said "would rename" while apply refused). (3) A held/became_tracked skip no longer consumes the batch_limit budget (only rows that actually mutate count), so a leading held album can no longer stall a batched resume. (4) A failed op that mutated nothing (e.g. an unapproved plan) is treated as a rollback no-op rather than a rollback failure. (5) Post-crash op recovery now matches the transaction's structured source instead of a JSON-substring query, so a folder name containing a quote or backslash is no longer silently missed.