Skip to content

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675

Open
indietyp wants to merge 7 commits intobm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and
Open

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675
indietyp wants to merge 7 commits intobm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and

Conversation

@indietyp
Copy link
Copy Markdown
Member

@indietyp indietyp commented Apr 29, 2026

🌟 What is the purpose of this PR?

Rework the MIR interpreter to support suspend/resume execution, richer value representations, and a proper input system. Together these let the interpreter pause when it needs external data (e.g. a database query), hand back a structured description of what it needs, and resume once the caller provides the result.

🔍 What does this change?

Suspension model (interpret/suspension/)

  • Introduce a suspend/resume protocol: when the interpreter hits a GraphRead terminator it yields a Suspension instead of blocking. Callers drive a start/resume loop until they get a Return.
  • GraphReadSuspension extracts temporal axes, filter parameters, and entity type from the suspended state.
  • Continuation carries the response value that resumes execution.
  • TemporalAxesInterval / Timestamp give structured representations of the temporal query window.
  • Runtime::run convenience wrapper for callers that can fulfill suspensions synchronously.

Input system (interpret/inputs.rs)

  • Add Inputs<'heap, A>: a typed map from interned symbols to Values, consulted for InputOp::Load and InputOp::Exists. Replaces the unstructured FastHashMap previously threaded through the runtime. Supports both global and custom allocators.

Value representation (interpret/value/)

  • Int: carry bit-width (1-bit boolean vs 128-bit integer) so serialization can distinguish true/false from 0/1 without external type info. Arithmetic promotes to 128-bit; bitwise boolean ops preserve 1-bit.
  • Struct: add StructBuilder<N> for stack-allocated fixed-field construction. Enforce sorted field invariant. Add merge and project_by_name.
  • Minor improvements to Opaque, List allocator parameterization, Str display.

SSA repair fix (pass/transform/ssa_repair)

  • Fix block parameters not being renamed during SSA repair.
  • Extend use-before-def detection to terminators, not just statements.

Entity path type resolution (pass/execution/traversal)

  • Add field_path and resolve_type to EntityPath for navigating from an entity path (e.g. WebId, Properties) to its concrete stdlib type.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

None

🐾 Next steps

  • The orchestrator in hashql-eval consumes the suspension model and input system introduced here.

🛡 What tests cover this?

  • Interpreter unit tests (interpret/tests.rs): cover the suspend/resume loop, input resolution, and value operations.
  • SSA repair UI tests (tests/ui/pass/ssa_repair): cover the block parameter and terminator fixes.
  • Existing pass UI tests re-blessed for snapshot changes from the Int representation change.

❓ How to test this?

  1. cargo nextest run --package hashql-mir
  2. cargo test --package hashql-mir --doc

feat: make Int size aware

feat: postgres serialization

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: errors

feat: psql codec

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: move bridge -> orchestrator

feat: move bridge -> orchestrator

feat: organize the runtime a bit more

feat: organize the runtime a bit more

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: organize the runtime a bit more

feat: finish orchestrator first version

feat: finish orchestrator first version

feat: feat: move out suspension requests to own module

chore: clippy

feat: split out into more manageable functions

feat: split out into more manageable functions

chore: orchestrator docs

feat: organize the runtime a bit more

feat: test orchestrator

feat: better symbol repr

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: test orchestrator

feat: pipeline

feat: bless snapshots

chore: tests

chore: masking

chore: tests

fix: SSA repair

chiore: ssa repair snapshots

feat: orchestrator test harness

chore: remove spec

fix: library features

chore: update snapshots

chore: update docs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:14 Inactive
@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 29, 2026

PR Summary

High Risk
Large refactor of the core interpreter runtime and value semantics, including a new suspension protocol and changes to integer/boolean representation, which can affect execution correctness and backend integration.

Overview
Adds a suspend/resume execution model to the HashQL MIR interpreter: GraphRead terminators now yield a typed Suspension (with temporal axes extraction) and resume via Continuation, plus new Runtime::{start,resume,run_until_suspension,run_until_transition}/Yield APIs and expanded callstack/frame state to support orchestration.

Replaces the runtime’s ad-hoc input map with an Inputs container (allocator-aware) and refactors interpreter error/diagnostic plumbing to be suspension-aware (new RuntimeError generic with suspension payload, new diagnostic category, and additional type-invariant errors).

Overhauls value handling: Int now tracks bit-width to distinguish booleans from integers (updating Postgres compilation and many MIR snapshots), Opaque/Value projections now transparently traverse wrapped values, and Struct enforces sorted fields with a new stack-based StructBuilder; tests/benches are updated and expanded accordingly. Execution-pass utilities also gain small APIs (e.g. island graph lookup) and traversal paths can now resolve their stdlib TypeId.

Reviewed by Cursor Bugbot for commit b2d61fe. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Apr 29, 2026 4:00pm
petrinaut Ready Ready Preview Apr 29, 2026 4:00pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Apr 29, 2026 4:00pm

@github-actions github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Apr 29, 2026
@indietyp indietyp changed the title HashQL: Suspendable interpreter with structured values and input system BE-523: HashQL: Suspendable interpreter with structured values and input system Apr 29, 2026
Copy link
Copy Markdown
Member Author

indietyp commented Apr 29, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 63.07%. Comparing base (deb5f19) to head (b2d61fe).

Files with missing lines Patch % Lines
libs/@local/hashql/eval/src/postgres/filter/mod.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@                                 Coverage Diff                                 @@
##           bm/be-522-hashql-expand-stdlib-type-definitions    #8675      +/-   ##
===================================================================================
- Coverage                                            63.19%   63.07%   -0.12%     
===================================================================================
  Files                                                 1338     1336       -2     
  Lines                                               138468   137229    -1239     
  Branches                                              5557     5529      -28     
===================================================================================
- Hits                                                 87499    86552     -947     
+ Misses                                               50047    49767     -280     
+ Partials                                               922      910      -12     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.40% <ø> (ø)
apps.hash-api 0.00% <ø> (ø)
local.hash-graph-sdk 9.63% <ø> (ø)
local.hash-isomorphic-utils 0.00% <ø> (ø)
rust.hash-graph-api 2.52% <ø> (ø)
rust.hashql-ast 87.23% <ø> (ø)
rust.hashql-compiletest 29.63% <ø> (ø)
rust.hashql-diagnostics 72.53% <100.00%> (+0.09%) ⬆️
rust.hashql-eval 81.08% <0.00%> (ø)
rust.hashql-hir 89.06% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:18 Inactive
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 29, 2026

Merging this PR will not alter performance

✅ 80 untouched benchmarks


Comparing bm/be-523-hashql-suspendable-interpreter-with-structured-values-and (b2d61fe) with bm/be-522-hashql-expand-stdlib-type-definitions (deb5f19)

Open in CodSpeed

Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:41 Inactive
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 15:09 Inactive
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/int.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Comment thread libs/@local/hashql/mir/src/interpret/value/mod.rs Fixed
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0a89a10. Configure here.

Comment thread libs/@local/hashql/mir/src/interpret/runtime.rs
@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Apr 29, 2026

🤖 Augment PR Summary

Summary: This PR upgrades the HashQL MIR interpreter to support structured runtime values and a suspend/resume execution model for external data access.

Changes:

  • Introduced a new Inputs map type and updated Runtime to borrow inputs rather than owning a hash map
  • Added interpreter yield points via Yield, plus Suspension/Continuation types to support suspend/resume around GraphRead
  • Extended runtime diagnostics/errors to model suspension-related failures and new invariant violations (unexpected value type / invalid constructor)
  • Reworked Int to track bit-width (bool vs 128-bit int) so boolean provenance round-trips cleanly through serialization
  • Improved value projection to transparently project into Opaque wrappers and added a StructBuilder helper for building sorted-field structs safely
  • Adjusted tests/benchmarks and various MIR passes (SSA repair, inst simplification, scheduling, traversal typing) to match the new runtime/value semantics

Technical Notes: Interpreter execution can now be driven synchronously via Runtime::run (with a suspension handler) or manually via Runtime::start/Runtime::resume, enabling backends like graph/DB to fulfill requests out-of-band.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

BinOp::BitAnd => return Some(lhs & rhs),
BinOp::BitOr => return Some(lhs | rhs),
// Comparisons produce booleans
BinOp::Eq => lhs.as_int() == rhs.as_int(),
Copy link
Copy Markdown

@augmentcode augmentcode Bot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libs/@local/hashql/mir/src/pass/transform/inst_simplify/mod.rs:262: eval_bin_op uses lhs.as_int()/rhs.as_int() for comparisons, which drops the new Int size/provenance (bool vs int) and makes const-folding potentially disagree with runtime comparisons. Consider using the Int comparison impls directly so folding matches interpreter behavior.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Comment thread libs/@local/hashql/mir/src/interpret/suspension/graph_read.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants