Skip to content

fix: include view data in pipeline responses and skip zero-delta commits#159

Merged
streamer45 merged 7 commits intovideofrom
devin/1773509119-fix-compositor-stale-data
Mar 14, 2026
Merged

fix: include view data in pipeline responses and skip zero-delta commits#159
streamer45 merged 7 commits intovideofrom
devin/1773509119-fix-compositor-stale-data

Conversation

@staging-devin-ai-integration
Copy link
Copy Markdown
Contributor

@staging-devin-ai-integration staging-devin-ai-integration bot commented Mar 14, 2026

Summary

Fixes a regression in the compositor node UI (Monitor view) where stale layout data appeared on the client and was sent back to the server on the next interaction (e.g. click-to-select).

Root cause — four interacting bugs:

A) Missing initial view data: The compositor server emits resolved layout on startup and the engine stores it in node_view_data, but pipeline API responses (both REST and WebSocket) never included this snapshot. New clients fell back to config-parsed positions which could disagree with the server's aspect-fit adjusted layout.

B) Zero-delta click commits stale data: handlePointerUp fired throttledConfigChange unconditionally — even for click-to-select with zero pointer movement. This sent stale config-parsed positions for ALL layers back to the server, overwriting the server's correct resolved layout.

C) "Sync from props" effect overwrites server geometry: When the server echoes back updated params, the "sync from props" effect in useCompositorLayers re-parses all layer positions from params via parseLayers(). For layers with rect: null (auto-layout), this produces {x:0, y:0, width:canvasWidth, height:canvasHeight} — overwriting the server's aspect-fit adjusted positions that useServerLayoutSync had applied. This caused a visible revert ~2 seconds after any drag.

D) preserveGeometry only preserved x/y/w/h — not all server-resolved fields: The initial fix (C) only kept x, y, width, height from existing state, but rebuilt the overlay by spreading { ...parsed, ... }. This lost runtime-only fields (measuredTextWidth, measuredTextHeight) and allowed stale param-derived values for opacity, rotationDegrees, zIndex, and mirror flags to overwrite the server's resolved state — causing text overlays to revert size/position on focus change.

Fix:

Server-side:

  • Add Session::get_node_view_data() to expose the engine's stored view data
  • Add Pipeline.view_data field (runtime-only, follows Node.state pattern)
  • Populate view_data in both REST and WebSocket get_pipeline handlers

Client-side:

  • Extract pipeline.view_data into sessionStore.nodeViewData on receipt (both setPipeline and batchSetPipelines)
  • Guard handlePointerUp: skip config commit when pointer delta is zero
  • Rewrite mergeOverlayState Monitor-view merge: when preserveGeometry is true, start from the full existing overlay (preserving ALL server-resolved OverlayBase fields + runtime measurements like measuredTextWidth), then overlay only type-specific config fields (text, fontSize, fontName, color, dataBase64) from parsed params. Uses pickConfigFields helper + OVERLAY_BASE_KEYS set to cleanly separate server-owned vs config-only fields.
  • Regenerated TypeScript types for the new Pipeline.view_data field

Tests:

  • Unit tests for mergeOverlayState preserveGeometry behavior (9 tests): OverlayBase field preservation, config field updates, runtime field retention, referential equality, image overlay support, new/removed overlay handling
  • Unit tests for setPipeline view_data extraction (initial load, merge, null, batch)
  • Unit tests for zero-delta pointer up guard (click does NOT commit, drag DOES commit)
  • Integration tests for Monitor view data flow (7 tests): server layout survives params echo-back, text measurements survive echo-back, config changes picked up while preserving server geometry, opacity/rotation/zIndex survive echo-back, focus changes don't reset video or text overlay positions, Design view control test

Review & Testing Checklist for Human

  • Verify with a live webcam PiP pipeline in Monitor view: Start a pipeline with a compositor node (e.g. webcam PiP), open the Monitor view in a separate tab/window. Layers should render at the correct server-resolved positions immediately, not jump on first load.
  • Click-to-select should not cause layout jumps: In the Monitor view, click on a layer to select it (without dragging). The layer positions should remain stable — no snapping to different coordinates.
  • Drag then wait — no revert: Drag a layer to a new position and wait 5+ seconds. The layer should stay in its new position — no revert to the original position.
  • Text and image overlays: Position text overlays and image overlays centrally, then wait. They should remain in position (not revert).
  • Focus change stability: Select different layers in sequence (text, video, image). No overlay should revert its size or position when losing/gaining focus.
  • Multiple clients: Open the Monitor view in two browser tabs. Change the layout in one and verify the other receives the correct positions (not stale data).

Notes

  • The pre-existing clippy lint failures in crates/nodes/src/video/compositor/kernel.rs and tests.rs are NOT introduced by this PR — they fail on the video branch without any changes.
  • The Pipeline.view_data field uses skip_serializing_if = "Option::is_none" and serde(default) so it's backward-compatible with existing pipeline definitions (YAML configs never include it).
  • The preserveGeometry flag only applies in Monitor view (when sessionId is set). Design view (no sessionId) continues to use config-parsed positions as the source of truth, which is the correct behavior for editing.

Link to Devin session: https://staging.itsdev.in/sessions/a5c78299b1204298b83f286a29918a25
Requested by: @streamer45


Staging: Open in Devin

streamkit-devin and others added 2 commits March 14, 2026 17:34
Root cause: two interacting bugs caused stale compositor layout data.

A) Missing initial view data: The compositor server emits resolved
   layout on startup, and the engine stores it in node_view_data, but
   pipeline API responses never included this snapshot. New clients
   fell back to config-parsed positions which could disagree with the
   server's aspect-fit adjusted layout.

B) Zero-delta click commits: handlePointerUp fired
   throttledConfigChange unconditionally — even for click-to-select
   with zero movement. This sent stale config-parsed positions for
   ALL layers, overwriting the server's correct resolved layout.

Server-side:
- Add Session::get_node_view_data() to expose engine's stored view data
- Add Pipeline.view_data field (runtime-only, like Node.state)
- Populate view_data in both REST and WebSocket get_pipeline handlers

Client-side:
- Extract pipeline.view_data into sessionStore.nodeViewData on receipt
- Guard handlePointerUp: skip config commit when pointer delta is zero
- Regenerate TypeScript types for the new Pipeline.view_data field

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
@staging-devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

staging-devin-ai-integration[bot]

This comment was marked as resolved.

streamkit-devin and others added 3 commits March 14, 2026 17:40
The 'sync from props' effect re-parses layers from params on every
params change. In Monitor view, this overwrites the server's resolved
layout (e.g. aspect-fit rects) with config-derived positions (e.g.
rect:null → full canvas), causing a visible revert ~2s after dragging.

Fix: add preserveGeometry option to mergeOverlayState. When sessionId
is set (Monitor view), existing layer geometry (x, y, width, height)
is preserved from current state (set by useServerLayoutSync) instead
of being overwritten by parsed params. New/removed layers still sync
correctly, and non-geometric fields (opacity, zIndex, visibility,
text content) continue to flow through from params.

Also fix test fixture: use camelCase property names matching
LayerState interface and include all required fields.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
The previous preserveGeometry fix only kept x, y, width, height from
existing state while rebuilding the overlay from parsed params.  This
lost runtime-only fields (measuredTextWidth, measuredTextHeight) and
allowed stale param-derived values for opacity, rotation, z-index,
and mirror flags to overwrite the server's resolved state.

Now when preserveGeometry is true (Monitor view), mergeOverlayState
starts from the full existing overlay and only overlays type-specific
config fields (text, fontSize, fontName, color, dataBase64) from the
parsed params.  This preserves all server-resolved spatial fields AND
any runtime measurements applied by useServerLayoutSync.

Adds unit tests for mergeOverlayState covering:
- OverlayBase field preservation from existing state
- Config field updates from parsed params
- Runtime field (measuredTextWidth) retention
- Referential equality when nothing changed
- Image overlay support
- New/removed overlay handling

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
Seven integration tests that exercise the full data flow between
useServerLayoutSync (server-driven layout) and the 'sync from props'
effect (config-driven state) in Monitor view:

1. Server-resolved layer positions survive params echo-back
2. Server text measurements (measuredTextWidth) survive echo-back
3. Config changes (text, fontSize) are picked up while preserving
   server geometry
4. Server-resolved opacity/rotation/zIndex survive echo-back
5. Layer focus changes do not reset video layer positions
6. Layer focus changes do not reset text overlay size/position
7. Design view (control) still uses parsed positions as source of truth

These tests would have caught all four variants of the stale data
regression (bugs A–D in #159).

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
staging-devin-ai-integration[bot]

This comment was marked as resolved.

streamkit-devin and others added 2 commits March 14, 2026 18:07
…r view

Without the comparator, dataBase64 changes from other clients are silently
dropped in Monitor view because the change-detection logic sees no OverlayBase
field differences and returns the old array unchanged.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
…itor view

Covers the bug where missing hasExtraChanges comparator caused dataBase64
updates from other clients to be silently dropped in Monitor view.

Signed-off-by: StreamKit Devin <devin@streamkit.dev>
Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
@streamer45 streamer45 merged commit 614eec2 into video Mar 14, 2026
1 check passed
@streamer45 streamer45 deleted the devin/1773509119-fix-compositor-stale-data branch March 14, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants