Skip to content

No way to know where a song's sections are without finding them by ear #484

Description

@thcp

The problem

StemDeck can tell you where every beat is, but nothing about where the song
goes. To loop the second chorus, or drop into the bridge, or hand a bandmate a
timestamp, you have to find the boundary by ear and drag a marker there
yourself, for every song, every time.

The sections bar already exists and is already editable. What is missing is
anything to put in it.

What makes this harder than it sounds

  • Model output is untrusted input. Whatever produces the suggestion is a
    third-party model. Its timestamps, its labels, and its structure all have to
    be validated before they touch a job, and a malformed result has to leave the
    separation exactly as it was.
  • It costs real time. Inference is a CPU pass measured in minutes. A user
    who only wanted stems should not silently pay for it, which means it needs to
    be a visible choice rather than an always-on cost.
  • The stems are the wrong shape. StemDeck separates six stems; the models
    in this space expect four. Mapping between them without running a second
    separation is a constraint on any solution.
  • Labels will be wrong sometimes. Functional names are not ground truth
    even when the boundaries are right, so whatever ships has to present them
    honestly and stay editable rather than claiming more confidence than it has.
  • Failure must be non-fatal. This is the last stage of a pipeline that has
    already done the expensive work. Nothing here may cost a user their
    separation, and cancellation has to stay responsive throughout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions