Skip to content

Drag stems and loop regions out to a DAW or a folder - #571

Merged
thcp merged 2 commits into
mainfrom
feat/drag-audio-out
Sep 4, 2026
Merged

Drag stems and loop regions out to a DAW or a folder#571
thcp merged 2 commits into
mainfrom
feat/drag-audio-out

Conversation

@thcp

@thcp thcp commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Closes #570

From discussion #474, item 4.

The problem

Every way audio left StemDeck ended in a save dialog: pick a format, click Export, find the folder. For someone auditioning loops against a session that is not one interruption, it is the whole workflow, repeated for every candidate.

Dragging is the gesture these users already have in their hands, and a DAW timeline is a drop target by default.

What it does

Mark a loop and every lane grows a handle tucked inside the right edge of the selection, drawn in that lane's own waveform colour. Drag one and you get that stem's slice. A wider handle on the top edge, in gold, carries the mix.

A lane handle gives the stem at unity gain, ignoring fader, mute and solo, exactly as the lane's download button already does. Mute and solo shape a mixdown, and the mix is the top handle's job. The mix handle greys out and stops being draggable when every lane is muted, because a drag has nowhere to show the message the Export menu shows.

Files land in an exports folder beside the stems folder, changeable in Settings.

Why the shell had to do it

A WebView cannot hand the OS a file. HTML5 dragstart carries text and, in Chromium only, a DownloadURL some file managers accept; a DAW wants a real path and no browser will produce one. So the gesture is cancelled in JavaScript and the platform drag is started in Rust. Desktop only, and the handles are hidden everywhere else rather than offering a gesture that would do nothing.

tauri-plugin-drag was not used. It exposes start_drag only as a #[command], so it would mean JavaScript naming the file path. That is the rule the existing exports are built on: download_to_path holds the destination in Rust behind an opaque token precisely so nothing in the WebView can write an arbitrary URL to an arbitrary location, and since the page is served over http Tauri treats it as a remote origin where app-defined commands are not ACL-gated. The drag crate is taken directly instead and StemDeck's own command resolves the folder, the same way open_url is hand-rolled rather than taking tauri-plugin-opener.

Three constraints that shaped it

  • A drag cannot wait for a render. It has to reach the OS while the button is down, and a region mixdown is an ffmpeg run. The mix is warmed on pointerup (which also catches a fader move, since that changes the cache key) and a lane on hover, both with Range: bytes=0-0 so it costs the render and not the transfer.
  • The file has to outlive the drop. Reaper references a dropped file where it lies, so it cannot come from the mixdown cache, which is pruned at 20 files / 500 MB and would eventually delete audio a project still points at.
  • The loop region is already interactive. Two separate pointer handlers turn a drag there into a move, a resize, or a new selection, and each calls preventDefault. Both drag handles are excluded from both.

Zoom to 10x

Included because it is the same surface. This is a data change, not a constant: peaks.json carried 1500 points, which backs about 5x on a full-width panel, and past the point count the bars repeat their neighbours instead of revealing anything. It now carries 3000, and the two constants name each other.

Tracks separated before this keep their 1500 points and stop gaining detail past 5x on the streaming path. Re-separating fixes it. The Web Audio path scans the decoded buffer itself and is sharp either way. Worth a line in the release notes.

Server

No change. GET /stems/{name}.wav and mixdown.{ext} with start/end already produced everything that gets dragged.

Six bugs found by using it

Each has a test where one can exist.

The region drag passed a relative URL, so validate_download_url rejected every one and the error went to a console a release build does not show one _absolute() for both paths
wireLoopDrag excluded .loop-region but not the lane handles, so grabbing one redefined the loop instead of dragging one selector, both gestures
Handles gated on the selection being wider than 2% of the track, so short loops never showed them gate removed, not tuned
The preview SVG had no xmlns, so the image never loaded and every drag carried the app badge text label instead
The handle lived in the row's innerHTML, which a redraw rewrites, so it vanished on the second wheel notch re-added after the art
A lane's region had no warm render warmed on hover

The 2% gate could not have been caught here: the fixture is 6 seconds and MIN_LOOP_SEC is 0.2s, so every loop it can make is already over 3%. That is noted in the spec.

Verification

Playwright 127 passed (109 before this work)
Rust unit tests 16
cargo clippy / fmt clean, Windows and Linux via WSL
pytest 944 passed, 13 pre-existing Windows environment failures, unrelated
ruff check / format clean
i18n complete across all eleven tables
uv.lock untouched, so existing desktop installs are still offered this release in-app

Verified by hand on Windows into Explorer and a DAW. The drag itself lands in another application and cannot be tested headlessly, so macOS and Linux still want a pass, and Reaper specifically, since it references rather than copies.

Before merging

The Unraid template pin is deliberately untouched. It tracks the latest published pre-release, so it should be bumped after a release is published with its assets, not at merge time.

Thales added 2 commits September 4, 2026 12:37
Getting audio out of StemDeck was three actions every time: pick a format,
click Export, find the folder in a system dialog. For someone auditioning
loops against a project that is not one interruption, it is the session.

A WebView cannot hand the OS a file. HTML5 dragstart carries text and, in
Chromium only, a DownloadURL some file managers accept; a DAW wants a real
path and no browser will produce one. So the gesture is cancelled in
JavaScript and the platform drag is started in Rust. Desktop only, because
nowhere else can do it at all, and the grip is hidden rather than offering a
gesture that would silently do nothing.

Nothing new on the server. GET /stems/{name}.wav and mixdown.{ext} with
start/end already produced exactly what has to be dragged.

Three things shaped the implementation:

- The path never crosses IPC. tauri-plugin-drag exposes start_drag only as a
  #[command], so using it would mean JavaScript naming the file to drag. That
  is the rule download_to_path exists to keep: the page is served over http
  and Tauri treats it as a remote origin, so app-defined commands are not
  ACL-gated and nothing else is holding. The drag crate is taken directly and
  StemDeck's own command resolves the folder, the way open_url is hand-rolled
  rather than taking tauri-plugin-opener.

- A drag cannot wait for a render. It has to reach the OS while the button is
  still down, and a region mixdown is an ffmpeg run, so rendering during the
  gesture means the button is released before the drag ever attaches. The
  region is warmed on pointerup instead, with Range: bytes=0-0, which costs
  the render and not the transfer. pointerup rather than a loop-change hook
  because moving a fader also changes the mixdown, and so the cache key.

- The file has to outlive the drop. Reaper references a dropped file where it
  lies, so it cannot come from the mixdown cache, which is pruned at 20 files
  / 500 MB and would eventually delete audio a project still points at. It
  goes to an exports folder nothing cleans up, beside the stems folder,
  changeable in Settings.

The grip is a separate element inside #loop-region and is excluded from
wireLoopRegionAdjust: an HTML5 drag and a pointer drag on one element fight,
and the region would slide away as it was dragged out. Region filenames carry
their bounds, unlike Export Region, because the Rust side reuses a file that
is already there and two loops of one song sharing a name would mean the
second drag handing over the first one's audio.

uv.lock is untouched, so existing desktop installs are still offered this
release in-app.

12 Rust unit tests over filename sanitising and folder resolution, 8 e2e
tests over everything on the JavaScript side of the platform call. The drag
itself lands in another application and is verified by hand per platform.

Closes #570
… back it

Follows the first pass on #570, after using it.

One handle above the lanes could not say which track it would produce, so it
was read as belonging to the first one. Every lane now carries its own handle,
tucked inside the right edge of the selection and drawn in that lane's own
waveform colour. The wide handle on the top edge stays, in gold, and is the
mix.

A lane handle gives that stem at unity gain, ignoring its fader, mute and
solo, exactly as the lane's download button does. Mute and solo shape a
mixdown, and the mix is what the top handle carries.

Zoom goes to 10x. That is a data change, not a constant: peaks.json carried
1500 points, which backs about 5x on a full-width panel, and past the point
count the bars repeat their neighbours instead of revealing anything. It now
carries 3000, and both constants name each other so they cannot drift.
Tracks separated before this keep their 1500 points and stop gaining detail
past 5x on the streaming path; the Web Audio path scans the decoded buffer
itself and is sharp either way.

The drag preview is the lane's name on a plate in its colour. The instrument
glyph was tried first and abandoned: these are 24px line icons, and at the
size a drag image is drawn the bass reads as a key and the kit as a face. A
word cannot be misread.

Six bugs found by using it, each with a test where one can exist:

- The region drag passed a relative URL. validate_download_url takes only
  http/https, so it rejected every one of them, and the error went to a
  console that a release build does not show. There is now a single
  _absolute() both the drag and the export path go through.
- wireLoopDrag turns a drag anywhere on the waves column into a new selection
  and calls preventDefault, which kills an HTML5 drag before dragstart fires.
  It excluded .loop-region, so the mix handle worked and the lane ones
  silently redefined the loop instead. Both are named in one selector now.
- The handles were gated on the selection being wider than 2% of the track,
  which is a gate on the fraction of the song and so never met by the short
  loops people actually work with. Removed rather than tuned; a pixel
  threshold would go stale, since zooming changes the rendered width without
  passing through that code.
- The preview SVG had no xmlns. Inline in the document the parser infers it;
  loaded through an img the file is parsed standalone and simply fails, so
  every lane drag quietly carried the app badge instead.
- The handle lived in the row's innerHTML, which a redraw rewrites, so it
  vanished on the second wheel notch.
- A lane's region had no warm render of its own. It is warmed on hover, which
  always precedes the grab, rather than warming all six on every loop change.

Verified: 127 e2e, 16 Rust unit tests, ruff and clippy clean on Windows and
Linux, i18n complete across all eleven tables, uv.lock untouched.

Closes #570
@thcp
thcp merged commit 37a2c5f into main Sep 4, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Getting audio out of StemDeck takes three actions, every time

1 participant