Skip to content

fix(flows): pin a parked run to the graph it was approved against - #5293

Merged
graycyrus merged 1 commit into
tinyhumansai:mainfrom
graycyrus:fix/flows-resume-graph-pin
Jul 31, 2026
Merged

fix(flows): pin a parked run to the graph it was approved against#5293
graycyrus merged 1 commit into
tinyhumansai:mainfrom
graycyrus:fix/flows-resume-graph-pin

Conversation

@graycyrus

@graycyrus graycyrus commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Stacked on #5286 (fix/flows-resume-run-lifecycle). This branch contains that PR's commit as its base. Review only the second commit, and merge after #5286.

Summary

  • Closes a stale-approval hole: a run parked for approval could be resumed against a rewritten graph, firing an action the user never approved.
  • Pins a canonical hash of the graph onto the run row at park time and verifies it at resume, failing closed on mismatch.
  • Legacy rows (parked before this guard) are allowed through with a warning, so upgrading mid-park cannot strand in-flight approvals.

Problem

A run parks pending_approval on a node — say send — and the user sees an approval card describing that node as it existed at park time.

flows_resume then loaded the flow's current graph and resumed the old checkpoint against it:

let compiled = tinyflows::compiler::compile(&flow.graph)

There was no version pin between the checkpoint and the persisted graph, and flows_update has no in-flight/pending-run guard. So if save_workflow (or any other update) rewrote that node's args/slug while the run sat parked, the resume fired whatever the new config does — under an approval the user never saw.

This is reachable inside a single authoring session: the workflow_builder agent holds both save_workflow and resume_flow_run.

The engine-compatibility gate does re-run at resume, but nothing checked that the graph was the one the checkpoint was actually taken from.

Solution

  • A canonical SHA-256 of the graph is persisted on the run row when it parks (flow_runs.graph_hash, added through the existing add_column_if_missing idiom). The digest is computed over a key-sorted serialization, so it is stable regardless of JSON key order.
  • At resume the hash is recomputed from the current graph and compared. On mismatch it fails closed: refuse, settle the run terminally with a clear reason, and drop the checkpoint — never execute.

Deliberately not doing: blocking flows_update while runs are parked. That would let a stale park hold a flow hostage for the entire TTL. Failing closed at resume is the chosen trade, and it is documented at the guard.

Migration safety: rows written before this guard read back as graph_hash IS NULL and are allowed through with a warning, so upgrading while a run is parked cannot strand an in-flight approval. Pinned by a test.

Note for reviewers: two pre-existing tests changed, legitimately

flows_resume_marks_an_incompatible_legacy_checkpoint_failed and flows_resume_marks_a_checkpoint_with_an_incompatible_saved_child_failed both simulate a "legacy incompatible graph" by swapping the graph after park. Under the new pin that swap is now caught first — correctly, since the graph genuinely did change. Their fixtures re-pin the hash to the swapped-in graph so the compatibility gate they exist to cover is still the thing being exercised.

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
  • Diff coverage ≥ 80% — refuse-on-change, succeed-when-unchanged, hash stability across key order, and the legacy NULL path are all covered; cargo test --lib openhuman::flows = 558 passed, 0 failed
  • Coverage matrix updated — N/A: security hardening of an existing path, no feature rows added/removed/renamed
  • All affected feature IDs from the matrix are listed under ## RelatedN/A: no matrix feature rows affected
  • No new external network dependencies introduced
  • Manual smoke checklist updated if this touches release-cut surfaces — N/A: no user-facing surface change beyond a refusal message
  • Linked issue closed via Closes #NNNN/A: found by code review, no tracking issue filed yet

Impact

  • Runtime/platform: Rust core only.
  • Schema: adds a nullable graph_hash TEXT column to flow_runs via the existing idempotent migration idiom. No backfill; NULL is a meaningful "legacy, unknown" value.
  • Behaviour: resuming a run whose flow was edited after parking now returns a clear error and settles the run, instead of silently executing the new action. That is the point of the change, and it is user-visible.
  • Dependencies: sha2 was already a direct dependency — no Cargo.toml/Cargo.lock change.

Related


AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

Validation Run

  • pnpm --filter openhuman-app format:check — N/A, no frontend files changed
  • pnpm typecheck — N/A, no TypeScript changed
  • Focused tests: GGML_NATIVE=OFF cargo test --lib openhuman::flows558 passed, 0 failed
  • Rust fmt/check (if changed): GGML_NATIVE=OFF cargo check clean; cargo fmt applied
  • Tauri fmt/check (if changed): N/A, app/src-tauri untouched

Validation Blocked

  • command: N/A
  • error: N/A
  • impact: N/A

Behavior Changes

  • Intended behavior change: a parked run is bound to the graph it was approved against; editing the flow invalidates the pending approval.
  • User-visible effect: resuming after editing the workflow returns "the workflow changed after this run was paused" instead of executing the rewritten node.

Parity Contract

  • Legacy behavior preserved: runs parked before this guard (graph_hash IS NULL) resume exactly as before, with a warning log — no in-flight approval is stranded by the upgrade.
  • Guard/fallback/dispatch parity checks: the engine-compatibility gate and the pending-gate validation are unchanged and still pinned by their own tests.

Duplicate / Superseded PR Handling

  • Duplicate PR(s): none
  • Canonical PR: this one
  • Resolution: N/A

Summary by CodeRabbit

  • New Features
    • Added protection for paused workflows by detecting changes before approval-based resumes.
    • Unchanged workflows resume normally, while outdated runs are safely canceled.
    • Existing runs without version information remain compatible.
  • Bug Fixes
    • Improved consistency when preserving or clearing paused-run state during completion and expiration.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 11 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 36383767-b938-459e-a843-9f668c72082a

📥 Commits

Reviewing files that changed from the base of the PR and between 04818d0 and f57a60f.

📒 Files selected for processing (7)
  • src/openhuman/flows/bus.rs
  • src/openhuman/flows/medulla_bridge_tests.rs
  • src/openhuman/flows/ops.rs
  • src/openhuman/flows/ops_tests.rs
  • src/openhuman/flows/store.rs
  • src/openhuman/flows/store_tests.rs
  • src/openhuman/flows/types.rs
📝 Walkthrough

Walkthrough

Flow runs now store a canonical SHA-256 hash for workflows paused at approval gates. Resume checks reject changed workflows, cancel stale runs, and remove checkpoints. Legacy runs without hashes remain resumable. Storage, types, finalization paths, and tests support the new field.

Changes

Approval resume graph pinning

Layer / File(s) Summary
Run hash contract and persistence
src/openhuman/flows/types.rs, src/openhuman/flows/store.rs, src/openhuman/flows/*_tests.rs
FlowRun includes an optional graph_hash. Storage migrates, selects, maps, and persists the nullable field. finish_flow_run accepts the graph hash. Fixtures pass explicit None values where no pin exists.
Pause and resume graph validation
src/openhuman/flows/ops.rs
Approval pauses hash the workflow and require_approval. Resume rejects mismatched hashes, cancels stale runs, drops checkpoints, and warns for legacy unpinned runs. Other finalization paths clear stale hashes.
Pinning and compatibility validation
src/openhuman/flows/ops_tests.rs
Tests cover changed, unchanged, and legacy graphs, canonical key ordering, approval configuration changes, checkpoint compatibility, terminal writes, cancellation, and TTL behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

Suggested labels: rust-core, bug

Suggested reviewers: senamakel

Poem

A rabbit pins the graph with care,

Hashes wait at gates in air.
Changed paths stop before they roam,
Old blank pins still find their home.
Checkpoints hop away when stale—
Safe resumes now leave a trail.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: pinning parked flow runs to the approved graph.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@graycyrus

Copy link
Copy Markdown
Contributor Author

Review follow-up: widened the pin to cover require_approval, closing a hole the graph-only hash left open.

require_approval is not cosmetic — it feeds workflow_origin(...), which becomes the AgentTurnOrigin for the entire resumed execution. TrustedAutomationSource::Workflow { require_approval: false } auto-allows every external_effect tool call, where true parks each one for its own human decision. And it is settable independently of the graph: flows_update(.., graph_json: None, require_approval: Some(false), ..) leaves .graph byte-identical.

So the original graph-only pin left the exact hole this guard exists to close, one level down: park at a gate → user approves → flip require_approval to false with the graph untouched (pin still matches, T-M1 passes cleanly) → on resume, every downstream outbound node that would have parked for its own approval now fires unattended, under an approval the user never gave.

The hash now covers {graph, require_approval}. Same fail-closed shape, same legacy-NULL migration path — only what counts as "the approved configuration" got wider. Pinned by graph_hash_covers_require_approval_not_just_the_graph.

559 flows tests pass, cargo fmt --check clean.

Worth a follow-up sweep (not done here): anything else read off flow between store::get_flow and with_origin(...) in flows_resume that influences authorization should arguably be in the pin too. graph and require_approval are the two that clearly do today.

@graycyrus
graycyrus force-pushed the fix/flows-resume-graph-pin branch 2 times, most recently from 11e39a0 to 04818d0 Compare July 31, 2026 05:59
@graycyrus
graycyrus marked this pull request as ready for review July 31, 2026 09:59
@graycyrus
graycyrus requested a review from a team July 31, 2026 09:59

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graycyrus has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@coderabbitai coderabbitai Bot added bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure. labels Jul 31, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 04818d0535

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/openhuman/flows/ops.rs Outdated
Comment on lines +5128 to +5132
finish_flow_run_row(
config,
thread_id,
flow_id,
"cancelled",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Claim the parked run before settling stale resumes

When a resume request has already loaded the approved graph and flipped the row to running, a second resume that read the row while it was still pending but sees a concurrent graph edit can enter this mismatch branch. finish_flow_run_row updates both running and pending_approval rows, so this path can cancel the first resume's active row and then drop its checkpoint while approved side effects are still executing; since the return value is ignored, it can also record last_status as cancelled after another writer won. Please claim the row before doing the hash refusal, or use a pending-only guarded update and only record/drop on success.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed — you both independently found the same real hazard, and it was a genuine miss on my part: I added exactly this guard to flows_cancel_run earlier in this batch (its ORDER MATTERS note) and then failed to apply the same rule to the new refusal path here.

The refusal now settles the row first and treats the guarded write as the authority. Only when it actually matched does it record_run and drop_checkpoint; otherwise it logs that another resume or cancel owns the run and leaves that runs status and checkpoint untouched. The refusal error is returned either way, since this resume view of the graph is stale and it must never proceed regardless of who owns the row.

Pinned by stale_approval_refusal_does_not_settle_a_run_another_resume_claimed, which claims the run via mark_run_resuming first and asserts the caller learns the writes real verdict rather than assuming the row was still parked.

Also corrected the compute_graph_hash doc that claimed a hash failure is treated like a legacy row with no pin. On the park side None stores no pin, but on the resume side the comparison is Some(expected) != None, so it refuses and drops the checkpoint. Failing closed there is right; the doc claiming fail-open was not.

595 flows tests pass, cargo fmt --check clean.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
src/openhuman/flows/ops.rs (1)

5945-5972: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider moving the hashing helpers into their own module.

compute_graph_hash and canonicalize_json form a self-contained, dependency-light unit: a WorkflowGraph plus a bool in, a hex digest out. ops.rs is already close to 6000 lines, and this PR adds roughly 150 more. Extracting these two functions into something like src/openhuman/flows/graph_pin.rs would move the pin logic and its unit tests next to each other and would shrink ops.rs slightly.

This is a mechanical move with no behavior change, so it can be deferred to a follow-up.

Based on coding guidelines: "Prefer Rust modules of approximately 500 lines or fewer and maintain small, single-responsibility Unix-style modules."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/openhuman/flows/ops.rs` around lines 5945 - 5972, Extract
compute_graph_hash and canonicalize_json from ops.rs into a dedicated graph-pin
module such as graph_pin.rs, preserving their existing behavior and signatures.
Move the related hashing unit tests alongside these helpers, update imports and
call sites to use the new module, and remove the original definitions from
ops.rs.

Source: Coding guidelines

src/openhuman/flows/store.rs (1)

753-785: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider a parameter struct for finish_flow_run.

finish_flow_run now takes eight positional parameters. The last two, error: Option<&str> and graph_hash: Option<&str>, share the same type and sit next to each other. A transposed call compiles silently and writes the refusal message into the pin column, or the hash into the user-visible error field. The same shape now repeats in finish_flow_run_row in src/openhuman/flows/ops.rs at lines 5834-5843.

All current call sites look correct. This is a defensive change against future edits, so it can be deferred.

One option is a small FlowRunFinish<'_> struct with named fields, passed by both finish_flow_run_row and finish_flow_run. Another lower-cost option is a newtype such as GraphPin<'a>(&'a str) so the two arguments stop being interchangeable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/openhuman/flows/store.rs` around lines 753 - 785, Defer this defensive
API change; no implementation update is required because all current
finish_flow_run and finish_flow_run_row call sites are correct. If addressed
later, replace the adjacent error and graph_hash positional arguments with named
fields in a shared FlowRunFinish struct used by both functions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/flows/ops.rs`:
- Around line 5940-5944: Correct the documentation near compute_graph_hash to
distinguish park-side and resume-side failures: retain the existing fail-closed
resume behavior when run_record.graph_hash is Some and hashing returns None,
which rejects/cancels the run and drops its checkpoint. Remove the claim that
resume falls back to unknown/allow, while preserving the explanation that None
on the park side means no pin is stored.
- Around line 5127-5151: Use the boolean returned by finish_flow_run_row in the
stale-approval refusal branch as the authority for cleanup. Only call
store::record_run and drop_checkpoint after the guarded write reports success;
if it does not, skip both operations. Preserve returning
GRAPH_CHANGED_SINCE_PARK_ERROR in either case, following the established
handling in flows_cancel_run.

---

Nitpick comments:
In `@src/openhuman/flows/ops.rs`:
- Around line 5945-5972: Extract compute_graph_hash and canonicalize_json from
ops.rs into a dedicated graph-pin module such as graph_pin.rs, preserving their
existing behavior and signatures. Move the related hashing unit tests alongside
these helpers, update imports and call sites to use the new module, and remove
the original definitions from ops.rs.

In `@src/openhuman/flows/store.rs`:
- Around line 753-785: Defer this defensive API change; no implementation update
is required because all current finish_flow_run and finish_flow_run_row call
sites are correct. If addressed later, replace the adjacent error and graph_hash
positional arguments with named fields in a shared FlowRunFinish struct used by
both functions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d07d09d1-9e9f-46ae-942a-7de80bc3784f

📥 Commits

Reviewing files that changed from the base of the PR and between d630f15 and 04818d0.

📒 Files selected for processing (7)
  • src/openhuman/flows/bus.rs
  • src/openhuman/flows/medulla_bridge_tests.rs
  • src/openhuman/flows/ops.rs
  • src/openhuman/flows/ops_tests.rs
  • src/openhuman/flows/store.rs
  • src/openhuman/flows/store_tests.rs
  • src/openhuman/flows/types.rs

Comment thread src/openhuman/flows/ops.rs
Comment thread src/openhuman/flows/ops.rs Outdated
@graycyrus
graycyrus force-pushed the fix/flows-resume-graph-pin branch from 04818d0 to c2778c0 Compare July 31, 2026 10:13

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graycyrus has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@graycyrus

Copy link
Copy Markdown
Contributor Author

Manual test status — partial, noted for the reviewer.

Confirmed by hand in a running app: a node-level requires_approval gate parks the run correctly (status="pending_approval", pending_approvals=1), which is the state this guard protects.

Not confirmed by hand: the refusal path itself. The parked run was never resumed in that session (zero flows_resume calls in the core log), so the graph-hash comparison — the actual behaviour this PR adds — was not exercised end to end through the UI.

It is covered by automated tests: resume_refuses_when_the_graph_changed_after_park (park → rewrite the node → resume must refuse and must not execute), resume_succeeds_when_the_graph_is_unchanged, graph_hash_covers_require_approval_not_just_the_graph, plus the legacy-NULL migration case. 594 flows tests pass.

Worth a reviewer's eye on one thing the manual attempt surfaced: it was not obvious in the UI where to approve a parked run as opposed to the save/enable pre-authorization card (#5248). They are different gates — the pre-auth card grants action classes up front, the park suspends a specific node mid-run — and conflating them is easy. If the parked-run approval is hard to reach, that is a UX gap worth its own issue rather than something this PR should absorb.

A run parks `pending_approval` on, say, a `send` node, and the user sees an
approval card describing that node as it existed at park time. `flows_resume`
then loaded the flow's CURRENT graph and resumed the old checkpoint against it,
with no version pin between the two — so if `save_workflow` (or any other
`flows_update`) rewrote that node's args or slug while the run sat parked, the
resume fired whatever the NEW config does, under an approval the user never saw.
The builder agent holds both `save_workflow` and `resume_flow_run`, so this was
reachable in one authoring session.

The engine-compatibility gate re-runs at resume, but nothing checked that the
graph was the one the checkpoint was actually taken from.

Fix: persist a canonical SHA-256 of the graph on the run row when it parks
(`flow_runs.graph_hash`, added via the existing `add_column_if_missing` idiom),
recompute it at resume, and on mismatch FAIL CLOSED — refuse, settle the run
terminally with a clear reason, and drop the checkpoint rather than execute.
Hashing is over a key-sorted serialization so the digest is stable regardless of
JSON key order.

Deliberately NOT blocking `flows_update` while runs are parked: that would let a
stale park hold a flow hostage for the full TTL. Failing closed at resume is the
chosen trade, and it is documented at the guard.

Migration safety: rows written before this guard read back as `graph_hash IS
NULL` and are allowed through with a warning, so upgrading mid-park cannot
strand in-flight approvals. Pinned by a test.

Two pre-existing tests legitimately broke: both simulate a "legacy incompatible
graph" by swapping the graph after park, which the new pin now catches first —
correctly, since the graph did change. Their fixtures re-pin the hash so the
compatibility gate they exist to cover is still reached.

`sha2` was already a direct dependency; no Cargo.toml/Cargo.lock change.
558 flows tests pass.
@graycyrus
graycyrus force-pushed the fix/flows-resume-graph-pin branch from c2778c0 to f57a60f Compare July 31, 2026 10:48

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

graycyrus has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

@graycyrus
graycyrus merged commit 89818fe into tinyhumansai:main Jul 31, 2026
20 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in Team Openhuman Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug rust-core Core Rust runtime in src/: CLI, core_server, shared infrastructure.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant