Skip to content

improvement(deployment): solve multiple client side races, and deployed state management issues#4502

Merged
icecrasher321 merged 8 commits intostagingfrom
improvement/deploy-state-machine
May 8, 2026
Merged

improvement(deployment): solve multiple client side races, and deployed state management issues#4502
icecrasher321 merged 8 commits intostagingfrom
improvement/deploy-state-machine

Conversation

@icecrasher321
Copy link
Copy Markdown
Collaborator

Summary

Makes deployment state server-canonical and race-safe: deploy/redeploy/undeploy now reconcile against persisted workflow state, use last-write-wins semantics, and move external provider side effects through an outbox instead of holding long DB transactions. It hardens background execution across multiple app tasks with durable outbox leases, idempotent schedule enqueueing for both database and Trigger.dev backends, version-pinned scheduled workflow execution, and safer webhook cleanup/retry behavior.

Type of Change

  • Other: Code Improvement

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented May 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 7, 2026 11:06pm

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 7, 2026

PR Summary

High Risk
Touches core deployment orchestration, schedule claiming/execution, and outbox lease/retry logic; regressions could cause missed/duplicated executions or incorrect deploy state/warnings surfaced to users.

Overview
Makes deployment operations more race-safe and server-canonical. The deploy UI now blocks deploy/redeploy/promote/load actions while local saves or remote reconciliation are in flight, serializes deploy actions per workflow, and proactively resyncs the local draft from the server after successful deploys to avoid stale state.

Shifts chat/form deployment routes to use performFullDeploy with request-aware error mapping and adds form rollback cleanup on deploy failure; it also forbids changing a chat deployment’s workflowId via update. Deployment/undeploy endpoints now consistently surface warnings and set maxDuration on long-running routes.

Hardens background processing: scheduled execution claims are now transactional with SKIP LOCKED, capped budgets, stale-claim recovery, and idempotent job IDs (supported by both DB and Trigger.dev backends). Schedule payloads are version-pinned (deploymentVersionId) and execution skips/release locks when deployments change.

Improves reliability of long-running workers by expanding outbox processing to include workflow deployment handlers, adding runtime budget controls, and changing handler timeouts to keep leases (avoiding overlapping retries) plus a new processOutboxEventById path for immediate post-commit processing. Function-block logging now records “display inputs” (pre-rewrite code) while executing with context-variable rewritten code.

Reviewed by Cursor Bugbot for commit f542637. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 7, 2026

Greptile Summary

This PR makes deployment state server-canonical and race-safe by moving all external side-effects (webhook setup, schedule creation, MCP tool sync) out of long-running transactions and into a durable transactional outbox. Deploy, redeploy, undeploy, and version-activate now write an outbox event atomically with the state change, process it immediately after commit, and fall back to the background cron worker if the inline attempt fails.

  • Outbox pattern for side-effects (deployment-outbox.ts): each step checks isActive before acting and re-enqueues a corrective event if a concurrent redeploy invalidates the work mid-flight.
  • Schedule execution hardened (schedule-execution.ts): schedules are pinned to a deploymentVersionId at enqueue time; executions skip if the pinned version is no longer active.
  • Client-side races closed (use-collaborative-workflow.ts, workflow-diff/store.ts, operation-queue/store.ts): remote updates are deferred while a diff is active or local operations are in-flight, then replayed via a monotonic reload-sequence counter once the queue drains.

Confidence Score: 5/5

Safe to merge — the outbox approach is well-structured and the inline-then-cron fallback makes side-effects durable even if immediate processing is preempted.

The core deploy/undeploy/activate flows now write state and queue side-effects atomically, eliminating the main class of races. Both findings are minor: an ignored lock return value that only matters for concurrently deleted workflows, and an unreferenced counter in the operation queue store. Neither affects correctness on the happy path.

apps/sim/lib/workflows/persistence/utils.ts — saveWorkflowToNormalizedTables acquires but ignores the lockWorkflowForUpdate result.

Important Files Changed

Filename Overview
apps/sim/lib/workflows/deployment-outbox.ts New file: transactional outbox for deployment side-effects (webhooks, schedules, MCP tools). Idempotent handlers with activation guards before each step.
apps/sim/lib/workflows/orchestration/deploy.ts Side-effects moved to outbox pattern; deploy/activate/undeploy now enqueue events transactionally and process them immediately after commit with cron-worker-safe locking.
apps/sim/lib/workflows/persistence/utils.ts Added lockWorkflowForUpdate and transaction-wrapped operations; saveWorkflowToNormalizedTables acquires the lock but silently ignores the return value.
apps/sim/lib/core/outbox/service.ts Added processOutboxEventById for immediate post-commit processing, plus separate timeout vs. error retry paths and deadline-aware batch processing.
apps/sim/background/schedule-execution.ts Added deployment-version pinning to schedule execution: schedules skip if the pinned version is no longer active.
apps/sim/stores/operation-queue/store.ts Added hasPendingOperations / waitForWorkflowOperations helpers and atomic dedup filter; workflowOperationVersions is tracked but never read.
apps/sim/hooks/use-collaborative-workflow.ts Hardened reloadWorkflowFromApi with reload-sequence guards and deferred-replay logic; self-consistent.
apps/sim/lib/webhooks/deploy.ts Split validation and save; added validateTriggerWebhookConfigForDeploy pre-flight check and isStillActive guard in cleanup paths.

Sequence Diagram

sequenceDiagram
    participant Client
    participant DeployAPI as Deploy API
    participant DB as Database (Tx)
    participant Outbox as Outbox Service
    participant CronWorker as Cron Worker
    participant Providers as Webhooks/Schedules/MCP

    Client->>DeployAPI: POST /deploy
    DeployAPI->>DB: BEGIN TRANSACTION
    DB->>DB: lockWorkflowForUpdate (FOR UPDATE)
    DB->>DB: validate schedules + webhooks
    DB->>DB: INSERT workflowDeploymentVersion
    DB->>DB: "UPDATE workflow.isDeployed = true"
    DB->>DB: INSERT outbox_event (SYNC_ACTIVE_SIDE_EFFECTS)
    DB->>DB: COMMIT
    DeployAPI->>Outbox: processOutboxEventById (inline, FOR UPDATE SKIP LOCKED)
    alt Inline processing succeeds
        Outbox->>Providers: saveTriggerWebhooks + createSchedules + syncMcpTools
        Outbox->>DB: INSERT outbox_event (CLEANUP_INACTIVE_SIDE_EFFECTS)
        DeployAPI->>Client: 200 OK
    else Lease held by cron or timeout
        DeployAPI->>Client: 200 OK (with warning)
        CronWorker->>Outbox: process pending outbox events
        Outbox->>Providers: saveTriggerWebhooks + createSchedules + syncMcpTools
    end
    Note over Outbox,Providers: Each step re-checks isActive before proceeding
Loading

Reviews (2): Last reviewed commit: "address comments" | Re-trigger Greptile

Comment thread apps/sim/lib/workflows/persistence/utils.ts
Comment thread apps/sim/app/api/schedules/execute/route.ts
Comment thread apps/sim/app/api/schedules/execute/route.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/app/api/form/route.ts
Comment thread apps/sim/app/api/form/route.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit f542637. Configure here.

@icecrasher321 icecrasher321 merged commit 1e5cc39 into staging May 8, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant