0.15.1 deletes existing library jobs from disk during startup

[diegoarusso]

Before reporting

• I am on the latest release and the issue still happens.
• I searched existing issues and did not find a duplicate.
• I stopped using the affected install and restored my library from a backup.

What happened?

After upgrading the Windows CPU-only portable build from 0.13.0 to 0.15.1,
starting the 0.15.1 backend removed existing completed jobs from both
data/jobs/registry.json and the filesystem.

This was reproduced twice. In both runs the library started with the same four
completed jobs and ended with only the two Hoobastank jobs.

Deleted:

Job ID	Source	Notes
5aabd06b8bde	Local file upload (prueba40.wav)	Had 8 separated stems plus the mix; the backup currently contains 9 WAV files in stems/.
7d6ce03419f2	YouTube (Mr. Big - To Be With You)	Completed job, about 249 MB; the backup contains 7 WAV files in stems/.

Survived:

• d81f3ab304ab — Hoobastank, imported from YouTube
• e0ed2aafc483 — Hoobastank, imported from YouTube

This was physical deletion, not only a library display or registry problem:
the two job directories disappeared from data/jobs/, and their entries
disappeared from registry.json.

Steps to reproduce

1. Start with StemDeck 0.15.1 portable and a restored data/jobs/ containing
   the four completed jobs listed above.
2. Confirm there are no existing StemDeck or backend processes and ports 8000
   and 8010 are free.
3. Start the 0.15.1 backend against that portable data directory.
4. Query /api/jobs, or stop the process and inspect data/jobs/ and
   registry.json.

Actual result: only the two Hoobastank jobs remain. The local-file job and the
Mr. Big job have been deleted from disk and from the registry.

Expected result: completed library jobs are preserved across startup and
upgrade.

Reproducibility and attribution

I reproduced this twice. During the first investigation I initially suspected
that my own process cleanup had removed the jobs. That attribution was wrong:
after restoring all four jobs again, the same two were deleted during a clean
startup with no prior processes or manual deletion involved.

I have stopped launching 0.15.1 and reverted to 0.13.0. The four original jobs
and stems are preserved in multiple backups, including a read-only copy outside
the application tree, so I can provide sanitized registry entries or compare
the two deleted jobs with the surviving jobs if that would help.

The rollback is verified, not only a filesystem swap: 0.13.0 opens normally
from the desktop shortcut and the application is usable again with the restored
library. This makes reverting to 0.13.0 a working temporary workaround for this
environment.

Environment

• Windows 11
• StemDeck 0.15.1
• StemDeck-Windows-x64.zip (CPU-only portable build)
• Intel i5-8350U, 16 GB RAM, Intel UHD 620
• Upgrade source: working 0.13.0 portable install

Not investigated

I did not continue starting 0.15.1 after confirming the second deletion because
this is destructive. I have not identified the cleanup code path or the exact
field difference that selects these two jobs. One potentially relevant
difference is that the local-file job had the additional lead/backing vocal
split, while the Mr. Big job was a normal YouTube import; therefore source type
alone does not explain the selection.

[thcp]

I reproduced this. The jobs really were deleted, from disk and from the registry, exactly as you describe.

The cause is the job TTL sweep. sweep_old_jobs removes completed jobs older than 24 hours, and it runs a few seconds after the backend starts. It is skipped when STEMDECK_DESKTOP=1, which StemDeck.exe sets on every launch, so the desktop app never sweeps a user's library.

I seeded a jobs directory with your four job IDs, two of them aged past the TTL, and started the 0.15.2 backend against it twice:

	before	after
without STEMDECK_DESKTOP=1	all four	d81f3ab304ab, e0ed2aafc483
with STEMDECK_DESKTOP=1	all four	all four

That is your result, down to which two survived.

It also answers the part you flagged as unexplained. The selection is age, not source type and not the vocal split. The two Hoobastank jobs were recent and the other two were not, so prueba40.wav and the Mr. Big job were the ones past the cutoff.

On the version: this is not new in 0.15.1. app/pipeline/collect.py has no commits between v0.13.0 and v0.15.1, and the sweep gate in app/main.py is byte identical across both. Your step 3 starts the backend directly, and port 8010 does not appear anywhere in StemDeck, so I think that run did not have the desktop flag set. Going back to 0.13.0 worked because it moved you back to launching through the app, which never sweeps.

So the mechanism is understood and your backups are safe to restore. Launch through StemDeck.exe and the library stays put.

I am leaving this open, because the honest reading is that you found a real problem. A destructive 24 hour default guarded only by an environment variable is too sharp an edge. Running the backend directly out of a portable package is a reasonable thing to do, and it should not quietly cost someone their library. That gate needs to be something better than an env var a direct launch silently omits.

Thank you for the quality of this report. Reproducing it twice, catching your own first attribution and correcting it, keeping a read-only copy outside the tree, and stopping rather than pushing on: that is what made this quick to pin down. Sorry it cost you the time and the scare.