Make StemDeck usable from another device

README.md

@@ -54,17 +54,20 @@ StemDeck is free and **does not accept any money, sponsorship, or funding**  fro
 | Category | Name | What they do | Link |
 |---|---|---|---|
 | Artists & Creators | Joao Gaspar | Producer, film scorer, touring/session musician | [@jay_glaspar](https://www.instagram.com/jay_glaspar) |
+| Artists & Creators | Killah Trakz | Producer | [@killahtrakz](https://www.instagram.com/killahtrakz/) |
 | Artists & Creators | More Notes Less Talk | Gear-focused creative project with a raw, tape-recorded identity | [@morenoteslesstalk](https://www.youtube.com/@morenoteslesstalk) |
+| Artists & Creators | Analog4Lyfe | Analog gear specialist | [@analog4lyfe](https://www.instagram.com/analog4lyfe) |
+| Artists & Creators | Dead röses | Cork-based punk rock band | [@dead_rosesband](https://www.instagram.com/dead_rosesband) |
 | Instrument Builders & Repair | Dlima Guitars | Custom guitars and basses | [@dlimaguitars](https://www.instagram.com/dlimaguitars) |
 | Instrument Builders & Repair | Lisbon Guitar Works | Handmade guitars in Lisbon | [dlimaguitars.com](https://dlimaguitars.com) |
 | Instrument Builders & Repair | Kris Luthier | Instrument repair and restoration | [@krisluthier](https://www.instagram.com/krisluthier) |
-| Music Gear | Analog4Lyfe | Analog gear specialist | [@analog4lyfe](https://www.instagram.com/analog4lyfe) |
 | Music Gear | Empress Effects | Boutique effects pedals | [empresseffects.com](https://empresseffects.com) |
 | Music Gear | Thomann | Large music-equipment retailer | [@thomann.music](https://www.instagram.com/thomann.music) |
 | Music & Karaoke Technology | Beltr | Local, subscription-free karaoke software | [beltr.app](https://beltr.app/) |
 | Music & Karaoke Technology | Seratone | TV-based karaoke system | [seratone.audio](https://seratone.audio/) |
 | Media & Community | slashCAM | Camera, video, and post-production media | [@slashcam.de](https://www.instagram.com/slashcam.de) |
 | Media & Community | r/bass | Bass-player community | [r/Bass](https://www.reddit.com/r/Bass) |
+| Writers & Storytellers | Alexandre Borges | Portuguese writer, screenwriter, and cultural commentator | [Books & author profile](https://www.instagram.com/alexgram_b/) |
 
 
 ---
@@ -336,9 +339,66 @@ The library is persistent by default (`STEMDECK_PERSIST_LIBRARY=1`), so tracks a
 | `STEMDECK_TIMEOUT_FFMPEG` | `300` | ffmpeg subprocess timeout (seconds). |
 | `STEMDECK_TIMEOUT_ANALYZE` | `120` | Audio analysis timeout (seconds). |
 | `STEMDECK_TIMEOUT_DEMUCS_STALL` | `1800` | Kill Demucs if no output for this many seconds. |
+| `STEMDECK_SSL_CERT` | (none) | PEM certificate; set with the key below to serve https directly. |
+| `STEMDECK_SSL_KEY` | (none) | PEM private key for the certificate above. |
+| `STEMDECK_HTTPS_PORT` | (none) | Serve https on this port *in addition* to the main listener. Set by the desktop app; see below. |
 
 `run.sh` also reads: `HOST` (default `127.0.0.1`), `PORT` (default `8765`), `RELOAD=1` (enable uvicorn auto-reload for development), `FOREGROUND=1` (run in foreground instead of backgrounding).
 
+### Serving other devices: why https is not optional
+
+Transpose is built on `AudioWorklet`, and browsers grant that only to a
+**secure context**. `https://` and `localhost` qualify. A plain
+`http://192.168.1.20:8000` does not, so a phone reaching StemDeck over plain
+http gets working playback and a key control that cannot do anything. There is
+no fallback worth shipping: driving the same DSP from a `ScriptProcessorNode`
+measured around 5% of the audio missing, because that node type drops buffers
+on its own at every size.
+
+So a server that other devices will use terminates TLS, one of three ways:
+
+1. **A reverse proxy** (SWAG, Nginx Proxy Manager, Traefik, Caddy). The usual
+   self-hosted shape, and the best one if you already run it. StemDeck reads
+   `X-Forwarded-Proto` and the RFC 7239 `Forwarded` header, so an https browser
+   over a plain-http upstream hop is recognised as secure and served normally.
+2. **StemDeck itself**, by pointing `STEMDECK_SSL_CERT` and `STEMDECK_SSL_KEY`
+   at a certificate and key. uvicorn serves them directly; no extra package is
+   installed for this.
+3. **A private overlay network** such as Tailscale, whose addresses are already
+   https.
+
+Reaching a plaintext non-local origin with none of those in place is refused
+with a 403 that explains this, rather than served as an app that is quietly
+half-broken. Loopback is always served, so turning this on can never lock the
+host out of its own server.
+
+### The desktop app runs two listeners
+
+The desktop app does the same thing without being configured, because it has
+two audiences that need opposite things.
+
+- **Plain http on `127.0.0.1`** for its own window. Loopback is already a
+  secure context, so nothing is lost, and it is the only scheme that works: a
+  self-signed certificate would raise a warning page the app window has no way
+  to click through.
+- **https on the LAN**, port 8443 by default, for phones and other computers.
+  This is the address Settings shows and the QR code points at.
+
+Both listeners serve the same process, so there is one library, one queue and
+one Demucs worker either way.
+
+The certificate is generated on your own machine the first time you enable
+network access, and lives in `<data>/certs/` beside `jobs/` and
+`settings.json`. Nothing is shipped in the download: a certificate in the
+release would publish its private key to everyone who downloaded it, which is
+worse than plain http because it looks secure. It is regenerated automatically
+when your machine's addresses change or the certificate is close to expiring.
+
+Because it is signed by nobody, **your phone will show a "your connection is
+not private" warning the first time**. Tap Advanced, then Continue. Once per
+device, per computer. Settings says so, in red, next to the toggle.
+
+
 ---
 
 ## API

app/api/jobs.py

@@ -34,6 +34,7 @@
 from app.core.registry import persist as registry_persist
 from app.core.registry import register_if_capacity as registry_register_if_capacity
 from app.core.registry import remove as registry_remove
+from app.core.registry import set_trashed as registry_set_trashed
 from app.core.settings import get_auto_sections, get_max_duration_sec
 from app.core.stems_location import is_relocating
 from app.pipeline import jobqueue
@@ -317,15 +318,47 @@ async def _create_local_job(request: Request) -> dict[str, str]:
 
 
 @router.get("")
-def list_jobs() -> list[dict]:
-    """List all completed jobs in the library, sorted by creation time."""
+def list_jobs(trashed: Literal["exclude", "include", "only"] = "exclude") -> list[dict]:
+    """Completed jobs in the library, oldest first.
+
+    Trashed jobs are left out by default, which is the whole point of the
+    parameter: this endpoint is the phone UI's entire library, and before the
+    Trash moved server-side it happily listed tracks the user had deleted on
+    their desktop hours earlier.
+    """
+    jobs = sorted(registry_all_jobs().values(), key=lambda j: j.created_at)
     return [
         _job_state(job)
-        for job in sorted(registry_all_jobs().values(), key=lambda j: j.created_at)
+        for job in jobs
         if job.status == "done"
+        and (trashed == "include" or (job.trashed_at is not None) == (trashed == "only"))
     ]
 
 
+@router.post("/{job_id}/trash")
+def trash_job(job_id: str) -> dict:
+    """Put a job in the Trash. Reversible, and nothing on disk is touched."""
+    if not JOB_ID_RE.match(job_id):
+        raise HTTPException(status_code=404, detail="job not found")
+    job = registry_set_trashed(job_id, True)
+    if job is None:
+        raise HTTPException(status_code=404, detail="job not found")
+    registry_persist(JOBS_DIR)
+    return {"job_id": job.id, "trashed_at": job.trashed_at}
+
+
+@router.post("/{job_id}/restore")
+def restore_job(job_id: str) -> dict:
+    """Take a job back out of the Trash."""
+    if not JOB_ID_RE.match(job_id):
+        raise HTTPException(status_code=404, detail="job not found")
+    job = registry_set_trashed(job_id, False)
+    if job is None:
+        raise HTTPException(status_code=404, detail="job not found")
+    registry_persist(JOBS_DIR)
+    return {"job_id": job.id, "trashed_at": job.trashed_at}
+
+
 @router.get("/{job_id}")
 def get_job(job_id: str) -> dict:
     """Get the current state of a job by ID."""

app/core/compression.py

@@ -0,0 +1,95 @@
+"""Compress the text StemDeck sends, and nothing else.
+
+Opening the phone UI pulls about 456 KB of JavaScript, CSS and HTML, of which
+`static/js/i18n.js` alone is 328 KB -- eleven language tables shipped to every
+user so that one of them can be read. Over loopback in the desktop webview that
+is invisible. Over Wi-Fi to a phone it is the load time, and it gets worse on
+https: browsers generally refuse to keep a disk cache for an origin with a
+certificate error, so the revalidation that would normally answer 304 fetches
+the whole thing again on every visit. Gzip takes that 456 KB to 126 KB.
+
+Starlette ships `GZipMiddleware` and it is nearly right: it already declines
+`text/event-stream`, so the job and queue streams keep flowing. What it does
+not do is look at the status code or the content type of anything else, and
+two of StemDeck's responses must not be touched.
+
+**Range responses.** The phone plays audio through
+`static/js/chunkedAudioEngine.js`, which asks for five-second windows with a
+`Range` header and gets `206 Partial Content` back. Compressing one rewrites
+`Content-Length` while leaving `Content-Range` describing the uncompressed
+bytes, and the two disagreeing is a specification corner browsers do not handle
+alike. The gain would have been nothing anyway: the payload is PCM.
+
+**Audio and video generally.** WAV does not compress, and paying deflate on a
+40 MB stem -- on the same machine that is running Demucs -- costs real time to
+save nothing.
+
+So the rule here is an allowlist by content type plus a hard `200`-only gate,
+rather than a list of paths, which would silently start compressing a stem the
+first time a route moved.
+"""
+
+from __future__ import annotations
+
+from starlette.datastructures import Headers
+from starlette.middleware.gzip import GZipMiddleware, GZipResponder
+from starlette.types import Message, Receive, Scope, Send
+
+# Everything StemDeck serves that is text under the hood. Matched as a prefix,
+# so the charset parameter ("text/html; charset=utf-8") does not need listing.
+COMPRESSIBLE_TYPES: tuple[str, ...] = (
+    "text/",
+    "application/javascript",
+    "application/json",
+    "application/manifest+json",
+    "image/svg+xml",
+)
+
+# Level 9 buys about 2% over level 6 on this content and costs several times the
+# CPU. The server doing this may also be separating a track.
+COMPRESS_LEVEL = 6
+
+# Below this a gzip header and trailer are most of what gets sent, and the round
+# trip dominates either way.
+MINIMUM_SIZE = 1024
+
+
+class _TextOnlyGZipResponder(GZipResponder):
+    """Starlette's responder, with a look at the response before committing.
+
+    The decision needs the status and content type, which only exist once the
+    application has answered, so it is made here on the way out rather than
+    from the request.
+    """
+
+    _pass_through = False
+
+    async def send_with_compression(self, message: Message) -> None:
+        if message["type"] == "http.response.start":
+            content_type = Headers(raw=message["headers"]).get("content-type", "")
+            # 200 only. That rules out 206 (a range window of audio) and 304
+            # (no body to compress), without naming either as a special case.
+            self._pass_through = message["status"] != 200 or not content_type.startswith(
+                COMPRESSIBLE_TYPES
+            )
+        if self._pass_through:
+            await self.send(message)
+            return
+        await super().send_with_compression(message)
+
+
+class TextGZipMiddleware(GZipMiddleware):
+    """`GZipMiddleware` restricted to text, and only when the client asked."""
+
+    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
+        if scope["type"] != "http" or "gzip" not in Headers(scope=scope).get("Accept-Encoding", ""):
+            # Starlette would run its IdentityResponder here purely to add a
+            # Vary header. Nothing between us and the browser on a LAN caches
+            # on our behalf, and buffering every response to add one header is
+            # not worth it.
+            await self.app(scope, receive, send)
+            return
+        responder = _TextOnlyGZipResponder(
+            self.app, self.minimum_size, compresslevel=self.compresslevel
+        )
+        await responder(scope, receive, send)

app/core/config.py

@@ -19,6 +19,12 @@ def _env_path(name: str, default: Path) -> Path:
     return Path(raw).expanduser().resolve() if raw else default
 
 
+def _env_path_opt(name: str) -> Path | None:
+    """A path setting with no default: absent means the feature is off."""
+    raw = os.environ.get(name, "").strip()
+    return Path(raw).expanduser().resolve() if raw else None
+
+
 def available_torch_devices() -> list[str]:
     """Compute devices this machine can actually use, best-first. CPU is always
     present; cuda/mps depend on the hardware + installed torch build. The
@@ -146,6 +152,32 @@ def _stored_jobs_dir() -> Path | None:
 MODELS_DIR = _env_path("STEMDECK_MODELS_DIR", DATA_DIR / "models")
 LOGS_DIR = _env_path("STEMDECK_LOGS_DIR", DATA_DIR / "logs")
 FFMPEG_DIR = _env_path("STEMDECK_FFMPEG_DIR", DATA_DIR / "ffmpeg")
+
+# ── TLS, for server deployments ───────────────────────────────────────────────
+#
+# AudioWorklet -- and so transpose -- is a [SecureContext] API, which browsers
+# grant to https:// and localhost and to nothing else. A phone on
+# http://<lan-ip> therefore cannot have it, and there is no workaround worth
+# shipping: driving the same DSP from a ScriptProcessorNode was measured
+# dropping ~5% of the audio, audibly, because that node type is lossy on its
+# own regardless of buffer size.
+#
+# Terminating TLS here costs nothing. uvicorn uses the stdlib `ssl` module, so
+# no package is added, uv.lock does not change, and the desktop in-app updater
+# is unaffected (see .claude/rules/desktop-update-gate.md). Generating a
+# certificate would need a dependency and would hand every client a full-page
+# browser warning, so StemDeck never does that: bring your own, from a reverse
+# proxy, Tailscale Serve, or mkcert.
+SSL_CERTFILE = _env_path_opt("STEMDECK_SSL_CERT")
+SSL_KEYFILE = _env_path_opt("STEMDECK_SSL_KEY")
+# The desktop app runs two listeners, not one: plain http on loopback for its
+# own webview, and https on the LAN for phones. It needs both because the two
+# have incompatible requirements -- the webview cannot be shown a certificate
+# warning it has no way to click through, and a phone cannot have transpose
+# without a secure origin. When this is set, app.main starts the second
+# listener alongside the first; when it is not, there is only ever one server
+# and TLS (if configured at all) belongs to whoever launched uvicorn.
+HTTPS_PORT = _env_int("STEMDECK_HTTPS_PORT", 0) or None
 FFMPEG_BIN = _env_path(
     "STEMDECK_FFMPEG",
     FFMPEG_DIR / ("ffmpeg.exe" if sys.platform.startswith("win") else "ffmpeg"),

app/core/models.py

@@ -98,6 +98,17 @@ class Job:
     # app/pipeline/vocal_split.py) and is recorded in stems/vocal_split_error.txt,
     # not job.error_detail, since the job itself did not fail.
     vocal_split: Literal["none", "running", "done", "error"] = "none"
+    # When the user put this job in the Trash, or None if they have not.
+    #
+    # Server-side on purpose. The Trash used to live only in the browser's
+    # catalog store, which is per-device: a track deleted on the desktop was
+    # still returned by GET /api/jobs, so the phone -- which builds its library
+    # straight from that endpoint -- listed everything the user thought they
+    # had thrown away. Two UIs, two answers to "what is in my library".
+    #
+    # A timestamp rather than a bool so the Trash can say when, and so a future
+    # auto-purge has something to work from.
+    trashed_at: float | None = None
     # Set by POST /api/jobs/{id}/cancel; consumed by pipeline stages.
     # Not surfaced via to_state() -- it's internal control state.
     cancel_requested: bool = False
@@ -151,6 +162,7 @@ def to_state(self) -> dict[str, Any]:
             "gpu_fallback": self.gpu_fallback,
             "stage_timings": self.stage_timings,
             "vocal_split": self.vocal_split,
+            "trashed_at": self.trashed_at,
             "created_at": self.created_at,
         }
 

app/core/registry.py

@@ -5,6 +5,7 @@
 import shutil
 import subprocess
 import threading
+import time
 import uuid
 from pathlib import Path
 
@@ -91,6 +92,21 @@ def remove(job_id: str) -> None:
         _procs.pop(job_id, None)
 
 
+def set_trashed(job_id: str, trashed: bool) -> Job | None:
+    """Move a job to the Trash, or take it back out.
+
+    Not a delete: the stems stay on disk and the job stays in the registry, so
+    restoring is free and the user's audio is never destroyed by a tap. Only
+    emptying the Trash calls DELETE, which is what actually removes files.
+    """
+    with _lock:
+        job = _jobs.get(job_id)
+        if job is None:
+            return None
+        job.trashed_at = time.time() if trashed else None
+        return job
+
+
 def all_jobs() -> dict[str, Job]:
     """Return a snapshot of the registry for sweep / cleanup."""
     with _lock: