`eve dev` local workflow store (`.workflow-data`) grows without bound; per-operation directory scans burn CPU under sustained workflow traffic

[Volpestyle]

eve version

eve@0.13.0 (observed); paths and behavior verified unchanged on main @ 1fd8a02 (@workflow/core 5.0.0-beta.26)

Node.js version

v26.3.1

Where does the bug occur?

Local dev (eve dev)

Description

The local workflow world persists every run, event, step, and hook as an individual JSON file under .workflow-data, and nothing ever deletes them. Combined with two scan behaviors, an always-on eve dev agent degrades steadily:

1. Per-operation directory scans in the local queue. @workflow/core's local world lists whole directories with suffix filters per queue operation (readdir + libuv scandir sort + per-filename V8 string internalization). Operation cost grows with lifetime accumulated files, not live work.
2. Boot-time full-store scan in eve itself. pruneDevelopmentRuntimeArtifactsSnapshots → collectWorkflowDataSnapshotPaths recursively readdirs and readFiles every file in .workflow-data (≤1MB each) to protect non-terminal runs' snapshots, so eve dev boot cost also grows with store size.

Observed impact

After roughly one week of an always-on agent (Discord bot + several coding agents driving it):

• .workflow-data: ~10,900 files / 315 MB — 8,466 events, 1,460 steps, 500 runs, 459 hooks
• 247 of 500 runs were stuck in running status forever — stranded by killed/restarted dev servers
• Under sustained workflow traffic the dev server held ~2.9 CPU cores; a 2s sample of the process is dominated by __getdirentries64, qsort/strcmp (libuv scandir), and V8 InternalizeString/GC — directory enumeration churn, not agent work
• A freshly restarted server idles at 0% over the same store, then burns again as soon as workflow traffic resumes
• After pruning stale runs externally (store → 12 MB / ~400 files), the same traffic runs quietly

Reproduction

1. Run eve dev for an agent that receives steady workflow traffic for days (or synthetically populate .workflow-data with ~10k run/event/step files in the local-world file format).
2. Drive concurrent workflow traffic.
3. sample <pid> / observe CPU: enumeration cost scales with total accumulated files.

Expected behavior

Dev-mode workflow history is bounded by a retention policy, and queue/boot cost does not scale with lifetime traffic.

Proposed direction

A boot-time background retention pass for terminal-status runs (completed / failed / cancelled), mirroring the existing pruneDevelopmentRuntimeArtifactsSnapshotsInBackground and pruneLocalSandboxTemplatesInBackground seams in startNitroDevelopmentServer: delete terminal runs older than a retention window together with their linked events, steps, hooks, and stream files. Non-terminal runs are never touched, so parked HITL runs survive (the property #172 established).

Two follow-on questions this does not answer, flagged for maintainer direction:

• Zombie running runs stranded by killed dev servers accumulate forever and are untouched by terminal-only retention. Should eve dev reconcile or expire them? They are exactly what Parked HITL turn crashes on resume with LoadCompiledManifestError after eve dev rebuild/restart GCs its snapshot #172 protects, so this needs a deliberate policy rather than an age cutoff.
• The per-operation O(N) scans in the local world are @workflow/core territory; retention bounds the pain but an index would fix the shape.

I have a PR implementing the terminal-run retention piece — happy to rework the approach per discussion here.

[Volpestyle]

I was doing a bunch of automated testing around my eve agent and my Mac fans started flipping out so I had Fable debug and it found the culprit was some runaway eve process. Might be a good idea to protect this so it can't happen.