[macOS] Cap 0.5.7 leaks ~17–18 MB/min after Studio recording/editor use until quit

[Tshah-95]

[macOS] Cap 0.5.7 leaks ~17–18 MB/min after Studio recording/editor use until quit

Description

On macOS 26.3, Cap 0.5.7 entered a stable, linear memory-growth state after several Studio recordings and editor use. The recording pipeline had finished successfully and Cap was no longer recording, but Cap's own once-per-minute process snapshots continued increasing by approximately 17–18 MB every minute until the process reached 13.1 GB RSS. A later vmmap sample reported a 25 GB physical footprint.

During this period, the machine developed brief system-wide stalls every 30–60 seconds. WindowServer memory also grew while Cap was in the leaking state. Cleanly quitting Cap stopped the growth; saved recordings were unaffected.

A controlled fresh-launch test did not reproduce the growth: with Cap launched and left untouched, RSS stabilized at 323–324 MB across three consecutive one-minute samples. This suggests that the leak requires retained state from recording/editor use rather than merely launching Cap or running its periodic process-memory sampler.

This may be related to #1589, but that report describes memory retained while a recording is paused. Here, the growth continued for many hours after recording tasks had completed and the app was idle.

Reproduction

This is the sequence that produced the issue; I have not yet reduced it to a single deterministic trigger:

1. Launch Cap 0.5.7.
2. Make several Studio window recordings using display capture, camera, microphone, and system audio.
3. Open/use the Studio editor and stop the recordings normally.
4. Leave Cap running after all recording tasks have finished.
5. Inspect the periodic process memory snapshot entries in Cap's log or monitor the process externally.

Expected: memory used by completed recordings/editor sessions is released and RSS stabilizes.

Actual: RSS increases by approximately 17–18 MB per minute until Cap is quit.

All observed capture, encoding, and muxing tasks logged successful completion or normal cancellation. The camera reported zero dropped frames, and the encoder did not time out.

Evidence

Selected Cap log snapshots from the leaking session:

2026-07-20 17:53:31  rss_mb=393
2026-07-20 17:54:31  rss_mb=411
2026-07-20 17:55:31  rss_mb=428
2026-07-20 17:56:31  rss_mb=382  # camera window closed just before this
2026-07-20 17:57:31  rss_mb=400
2026-07-20 19:00:31  rss_mb=1507
2026-07-20 23:59:31  rss_mb=7773
2026-07-21 23:59:31  rss_mb=8011  # machine slept for part of this interval
2026-07-22 15:14:50  rss_mb=13135 # clean app exit begins

External inspection near the end of the session:

Cap RSS:                 ~13.4 GB
Cap physical footprint:  25.0 GB (`vmmap`)

Fresh-launch idle control on the same machine and version:

2026-07-22 15:26:07  rss_mb=323
2026-07-22 15:27:07  rss_mb=323
2026-07-22 15:28:07  rss_mb=324

Cap's frame WebSocket continued reporting approximately 30 FPS / 17.7–18.0 MB/s during the fresh idle control, while RSS remained flat. That stream therefore does not appear sufficient by itself to cause the leak, though retained recording/editor consumers or cleanup may be relevant.

Additional context

• Cap: 0.5.7
• macOS: 26.3 (25D125)
• Hardware: Apple M4 Max, 128 GB RAM
• Display: Samsung C49RG9x, 3840×1080 at 60 Hz
• Camera: Logitech HD Pro Webcam C920, 1280×720 at 30 FPS
• Recording mode: Studio, with display/window capture, camera, microphone, and system audio

The complete logs contain local recording titles and filesystem paths, so I have included sanitized excerpts here. I can provide a more focused diagnostic extract or run a specific reproduction matrix if maintainers identify the most useful state transitions to test.

[linear]

CAP-761

[tembo]

This is a well-documented report. A few pointers that might help investigation:

Related issues:

• Excessive RAM usage (~15GB) #1589 describes similar high memory retention (~15GB) but specifically when a recording is paused. Your case of continuous growth after recording completion appears to be a distinct leak path, though both may share underlying causes around incomplete cleanup of recording/editor state.
• Reduce CPU / memory usage whilst recording #109 is the umbrella tracking issue for memory/CPU optimization during recording.

Potentially relevant code paths:

The linear ~17-18 MB/min growth rate while idle suggests something is accumulating frame data or buffers without release. Key areas to examine:

1. Editor instance cleanup (apps/desktop/src-tauri/src/editor_window.rs:211-224): The EditorInstance::dispose() cancels the WebSocket shutdown token and unlistens from render frame events. The Drop impl spawns async cleanup but catches panics if the runtime is unavailable (line 864-866 in crates/editor/src/editor_instance.rs). If disposal isn't called explicitly before the editor window closes, cleanup may be incomplete.

2. Camera preview manager (apps/desktop/src-tauri/src/camera.rs:376-388): begin_shutdown() takes the preview and sends a shutdown event, but if something holds a reference to InitializedCameraPreview or its camera_tx channel, the preview runtime thread may not fully exit.

3. WebSocket frame buffer (apps/desktop/src-tauri/src/frame_ws.rs): The frame WebSocket uses Arc<Vec<u8>> for frame data (WSFrame.data). If consumers (editor preview, camera preview) hold stale subscriptions, the broadcast channel could retain frames.

Recent related work:

PR #1669 (merged March 2026) addressed "improve WebSocket cleanup and disposal" and added desktop memory soak test scripts. The apps/desktop/src/utils/socket.ts file has dispose() and cleanup() methods (lines 373, 988) that null out WebSocket handlers and terminate workers. If the Solid.js cleanup isn't triggered when navigating away from the editor, consumers may leak.

Debugging suggestions:

Since your fresh-launch control shows stable memory, the leak is definitely tied to recording/editor lifecycle. A focused reproduction might be:

1. Record a single short Studio clip
2. Open the editor
3. Close the editor window (not the whole app)
4. Monitor if RSS grows from that point

If memory grows after step 3, the editor disposal path is incomplete. If it only grows while the editor is open but idle, something in the preview render loop may be accumulating.