Skip to content

feat: add single-threaded event loop for Promise.all and async array callbacks#33

Merged
TheUncharted merged 3 commits intomasterfrom
develop
Mar 13, 2026
Merged

feat: add single-threaded event loop for Promise.all and async array callbacks#33
TheUncharted merged 3 commits intomasterfrom
develop

Conversation

@TheUncharted
Copy link
Copy Markdown
Owner

@TheUncharted TheUncharted commented Mar 13, 2026

Summary

AI-generated code like Promise.all(arr.map(async fn => await external())) previously crashed the VM because Rust's for-loop couldn't be suspended mid-iteration. Inspired by Node.js and Python asyncio, the VM now has a continuation-based event loop that handles async callbacks in .map() and .forEach() — each await suspends and resumes sequentially, no threads needed.

Changes

  • Add Continuation enum to the VM with ArrayMap and ArrayForEach variants
  • Implement process_continuation() — fires when a callback frame pops, collects results, sets up next callback or finalizes
  • Track callback_frame_index to precisely detect when a callback completes (not just any frame at same depth)
  • Handle rejected promise unwrapping in continuation results
  • Serialize/deserialize continuation state in snapshots (postcard)
  • Add async guard error for unsupported methods (filter, find, reduce, etc.)
  • Update README: benchmarks, "Can do" / "Cannot do", supported syntax table
  • Add async map examples to Rust, Python, and TypeScript basic examples

Test plan

  • test_array_map_async_callback_with_external — 3 elements, 3 sequential suspensions
  • test_array_map_async_empty — empty array returns immediately
  • test_array_map_async_single_element — single element edge case
  • test_array_map_sync_still_works — regression test
  • test_sequential_external_calls_in_loop — sequential awaits pattern
  • async_map_3 benchmark added (11.6 µs median)
  • All existing tests pass (cargo test)

Summary by CodeRabbit

  • New Features

    • Async callbacks supported in .map() and .forEach() with per-item suspend/resume and improved Promise semantics.
  • Documentation

    • README clarifies supported Promise methods (.then, .catch, .finally, Promise.all) and marks async callbacks in map/forEach as supported; adds an async .map() benchmark.
  • Tests & Examples

    • Added comprehensive async/await tests and Python, Rust, and TypeScript examples demonstrating async array operations with external calls.

…e the VM

AI-generated code like `Promise.all(arr.map(async fn => await external()))` previously
crashed the VM because Rust's for-loop couldn't be suspended mid-iteration. Users can
now use async array callbacks with external function calls — each await suspends and
resumes sequentially via a continuation-based execution model.
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 13, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ee9c812f-f6d8-4289-b0e5-fb00b819b76d

📥 Commits

Reviewing files that changed from the base of the PR and between 443e163 and 4794aef.

📒 Files selected for processing (2)
  • crates/zapcode-core/src/vm/mod.rs
  • crates/zapcode-core/tests/async_await.rs

📝 Walkthrough

Walkthrough

Adds a continuation mechanism to the VM to support async callbacks in array methods (e.g., .map(), .forEach()), persists continuations in snapshots, updates execution/resume flow to process continuations, and adds benches, tests, and examples exercising per-element suspension/resume.

Changes

Cohort / File(s) Summary
Core VM Continuation System
crates/zapcode-core/src/vm/mod.rs
Add Continuation enum and Vm.continuations field; thread continuations through Vm::new/from_snapshot; implement process_continuation, is_async_callback, start_continuation_map / start_continuation_foreach; update execute_array_callback_method to return Result<Option<Value>> and drive async map/forEach via continuations; disallow async callbacks for other array methods.
Snapshot Serialization
crates/zapcode-core/src/snapshot.rs
Add continuations: Vec<Continuation> to VmSnapshot; capture vm.continuations.clone() and restore via Vm::from_snapshot parameter.
Tests
crates/zapcode-core/tests/async_await.rs
Add comprehensive tests for async/await and continuation flows: async map/forEach with external calls, empty/single-element cases, regression for sync map, and error tests for unsupported async callbacks in other array methods.
Benchmarks
crates/zapcode-core/benches/execution.rs
Add async_map_3 benchmark measuring async .map() (3 elements) with awaited inner promises.
Examples
examples/typescript/basic/main.ts, examples/rust/basic/basic.rs, examples/python/basic/main.py
Add scenarios demonstrating async map with external calls and per-item suspension/resume sequences; mock/resume external calls and show aggregated results.
Documentation
README.md
Update benchmark row for Async .map() (3 elements) timing; expand Supported Syntax to explicitly mark async callbacks in .map() / .forEach() as supported and list Promise methods (.then(), .catch(), .finally(), Promise.all).

Sequence Diagram

sequenceDiagram
    participant Client as Client
    participant VM as VM
    participant ContinuationStack as Continuation\r\nStack
    participant Callback as Callback

    Client->>VM: execute_array_map([item1, item2], asyncCallback)
    activate VM
    VM->>VM: detect async callback
    VM->>ContinuationStack: start_continuation_map(callback, [item1, item2])
    activate ContinuationStack
    ContinuationStack->>Callback: invoke(item1)
    activate Callback
    Callback->>VM: await external call (suspend)
    deactivate Callback
    VM->>Client: suspend (external call pending)
    deactivate VM

    Client->>VM: resume with result1
    activate VM
    VM->>ContinuationStack: process_continuation()
    ContinuationStack->>ContinuationStack: store result1, advance index
    ContinuationStack->>Callback: invoke(item2)
    activate Callback
    Callback->>VM: await external call (suspend)
    deactivate Callback
    VM->>Client: suspend (external call pending)
    deactivate VM

    Client->>VM: resume with result2
    activate VM
    VM->>ContinuationStack: process_continuation()
    ContinuationStack->>ContinuationStack: store result2, complete
    ContinuationStack->>VM: return aggregated results
    VM->>Client: complete with aggregated results
    deactivate VM
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding a continuation-based event loop for async array callbacks and Promise.all support.
Description check ✅ Passed The description follows the template with Summary, Changes, and Test plan sections clearly filled out; all required sections are present and substantive.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch develop
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 13, 2026

Benchmark Results

    Updating crates.io index
   Compiling zapcode-core v1.4.0 (/home/runner/work/zapcode/zapcode/crates/zapcode-core)
    Finished `bench` profile [optimized] target(s) in 8.76s
     Running unittests src/lib.rs (target/release/deps/zapcode_core-b075bfc4ac855948)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running benches/execution.rs (target/release/deps/execution-b4cebed678edeb20)
Timer precision: 20 ns
execution                  fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ array_creation          3.777 µs      │ 86.16 µs      │ 3.951 µs      │ 5.241 µs      │ 100     │ 100
├─ async_map_3             13.85 µs      │ 68.2 µs       │ 14.24 µs      │ 15.39 µs      │ 100     │ 100
├─ fibonacci_10            179.8 µs      │ 238.6 µs      │ 183 µs        │ 185.8 µs      │ 100     │ 100
├─ function_call           6.812 µs      │ 19.54 µs      │ 7.053 µs      │ 7.433 µs      │ 100     │ 100
├─ loop_100                105.2 µs      │ 154.3 µs      │ 106.4 µs      │ 109.6 µs      │ 100     │ 100
├─ object_creation         7.323 µs      │ 26.32 µs      │ 7.568 µs      │ 7.999 µs      │ 100     │ 100
├─ promise_all_3           9.827 µs      │ 31.23 µs      │ 10.12 µs      │ 10.82 µs      │ 100     │ 100
├─ promise_catch_resolved  6.451 µs      │ 33.77 µs      │ 6.587 µs      │ 6.903 µs      │ 100     │ 100
├─ promise_resolve_await   4.637 µs      │ 28.22 µs      │ 4.748 µs      │ 5.036 µs      │ 100     │ 100
├─ promise_then_chain_3    13.9 µs       │ 40.41 µs      │ 14.53 µs      │ 15.18 µs      │ 100     │ 100
├─ promise_then_single     7.944 µs      │ 22.98 µs      │ 8.184 µs      │ 8.398 µs      │ 100     │ 100
├─ simple_expression       3.416 µs      │ 16.07 µs      │ 3.516 µs      │ 3.819 µs      │ 100     │ 100
├─ string_concat           4.046 µs      │ 11.75 µs      │ 4.197 µs      │ 4.359 µs      │ 100     │ 100
├─ template_literal        4.507 µs      │ 17.53 µs      │ 4.648 µs      │ 4.97 µs       │ 100     │ 100
╰─ variable_arithmetic     4.517 µs      │ 12.93 µs      │ 4.668 µs      │ 4.845 µs      │ 100     │ 100


Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/zapcode-core/src/vm/mod.rs`:
- Around line 397-414: process_continuation currently only checks that
callback_frame_index was popped (by comparing self.frames.len()), but you must
also ensure execution has returned to the original caller frame depth to avoid
advancing stale continuations on unwinds; update the match arms for
Continuation::ArrayMap and Continuation::ArrayForEach to also extract the saved
caller_frame_depth and then only proceed when self.frames.len() ==
caller_frame_depth (otherwise return Ok(false)), ensuring both the
callback_frame_index check and the caller_frame_depth check gate continuation
completion.
- Around line 417-418: The line assigning callback_result uses
self.pop().unwrap_or(Value::Undefined) which masks stack underflow; change it to
surface an error instead of defaulting to Value::Undefined by checking
self.pop() and returning or propagating a stack-underflow error (e.g. convert to
self.pop().ok_or(VMError::StackUnderflow) or similar) so the caller of the
containing function (where callback_result is computed) receives a Result/Err on
underflow; adjust the containing function's signature/propagation to return that
error rather than silently using Value::Undefined.
- Around line 419-437: The current unwrapping in the callback result handling
(inside mod.rs where callback_result is inspected and self.continuations.pop()
is used on rejection) incorrectly treats any object with a "status" field as an
internal promise; update the check to only unwrap objects that have the internal
promise marker (e.g. require a __promise__ boolean flag alongside status) before
treating as {status: "resolved"/"rejected"} so ordinary user objects are left
untouched, and keep the existing behavior of popping continuations and returning
ZapcodeError::RuntimeError on rejected internal promises.

In `@crates/zapcode-core/tests/async_await.rs`:
- Around line 653-865: Add new tests alongside the existing map tests: a
positive test function (e.g. test_array_for_each_async_callback_with_external)
that uses items.forEach(async (item) => { await fetchData(item); }) with
start_with_externals("fetchData") and asserts the sequence of suspended external
calls and final VmState::Complete (and any side-effect results if forEach
returns undefined), an edge-case test (e.g. test_array_for_each_async_empty)
where items: string[] = [] that asserts immediate VmState::Complete and correct
empty/no-op behavior, and a negative test (e.g.
test_array_async_unsupported_methods_filter_reduce) that evaluates code using
async callbacks with .filter() and .reduce() and asserts the VM returns the
explicit guard error (match VmState::Complete(Value::Error(...)) or appropriate
panic expectations) to cover the guard paths introduced for async callbacks;
follow the naming pattern used by existing tests (test_array_map_*) and include
a sandbox-escape case if applicable per guidelines.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c3c03731-c608-4999-ab4b-2907771528b7

📥 Commits

Reviewing files that changed from the base of the PR and between a776506 and e245133.

📒 Files selected for processing (8)
  • README.md
  • crates/zapcode-core/benches/execution.rs
  • crates/zapcode-core/src/snapshot.rs
  • crates/zapcode-core/src/vm/mod.rs
  • crates/zapcode-core/tests/async_await.rs
  • examples/python/basic/main.py
  • examples/rust/basic/basic.rs
  • examples/typescript/basic/main.ts

Comment on lines +417 to +418
let callback_result = self.pop().unwrap_or(Value::Undefined);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Do not swallow stack underflow when reading callback return values.

Using unwrap_or(Value::Undefined) here masks stack-discipline bugs and can silently produce wrong map/forEach outputs.

Suggested fix
-        let callback_result = self.pop().unwrap_or(Value::Undefined);
+        let callback_result = self.pop()?;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let callback_result = self.pop().unwrap_or(Value::Undefined);
let callback_result = self.pop()?;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/zapcode-core/src/vm/mod.rs` around lines 417 - 418, The line assigning
callback_result uses self.pop().unwrap_or(Value::Undefined) which masks stack
underflow; change it to surface an error instead of defaulting to
Value::Undefined by checking self.pop() and returning or propagating a
stack-underflow error (e.g. convert to self.pop().ok_or(VMError::StackUnderflow)
or similar) so the caller of the containing function (where callback_result is
computed) receives a Result/Err on underflow; adjust the containing function's
signature/propagation to return that error rather than silently using
Value::Undefined.

User objects like {status: "resolved", value: 42} were incorrectly unwrapped
as internal promises — now only objects with the __promise__ marker are treated
as promises, preventing silent data corruption in async map callbacks.

Also guard against stale continuations firing on stack unwinds by checking
caller_frame_depth alongside callback_frame_index.

Tests added for forEach async callbacks, async guard errors on filter/reduce,
and the user-object unwrap fix.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
crates/zapcode-core/src/vm/mod.rs (1)

425-425: ⚠️ Potential issue | 🟠 Major

Stack underflow should propagate as an error, not default to Undefined.

Using unwrap_or(Value::Undefined) masks stack-discipline bugs and can silently produce incorrect map/forEach results.

Suggested fix
-        let callback_result = self.pop().unwrap_or(Value::Undefined);
+        let callback_result = self.pop()?;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/zapcode-core/src/vm/mod.rs` at line 425, The code currently masks
stack underflow by using let callback_result =
self.pop().unwrap_or(Value::Undefined);; change this to propagate an error
instead: replace unwrap_or with .ok_or(...) or .ok_or_else(...) to return a VM
error (e.g. VMError::StackUnderflow) and use the ? operator so the surrounding
function returns Result; specifically update the assignment of callback_result
to something like let callback_result =
self.pop().ok_or(VMError::StackUnderflow)?; and ensure the enclosing function
signature and error type support returning that error.
🧹 Nitpick comments (1)
crates/zapcode-core/src/vm/mod.rs (1)

658-665: Consider adding a resource limit check for continuation depth.

The continuations vector grows without an explicit limit check. While it's implicitly bounded by stack depth, adding a guard would be consistent with other resource-limit enforcement in the VM.

Example check
+        // Guard against deeply nested continuations
+        if self.continuations.len() >= 1024 {
+            return Err(ZapcodeError::RuntimeError(
+                "continuation depth limit exceeded".to_string(),
+            ));
+        }
+
         self.continuations.push(Continuation::ArrayMap {

As per coding guidelines: "Resource limits (memory, time, stack depth, allocation count) must be enforced during execution."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/zapcode-core/src/vm/mod.rs` around lines 658 - 665, Before pushing the
new Continuation::ArrayMap onto self.continuations, add a guard that checks the
current continuation depth against the VM's configured resource limit (e.g. a
max_continuations or continuation_depth limit on self or self.resource_limits);
if the limit is reached return an appropriate error/trap (same error type used
elsewhere for resource limits) instead of pushing, and only push after
incrementing/validating the count. Apply this check at the push site where
Continuation::ArrayMap { callback, source: arr, ... } is created so it
consistently enforces continuation depth like other VM resource limits.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/zapcode-core/src/vm/mod.rs`:
- Around line 431-448: The block using matches!(map.get("__promise__"),
Some(Value::Bool(true))) and the following else { match map.get("status") { ...
} } is misformatted; run rustfmt (cargo fmt) or reflow the block so it follows
rustfmt rules (e.g., proper brace placement and spacing) — adjust the
conditional/match formatting around the map.get("status") match arm and ensure
removal of any stray spacing so lines with map.get("value"), map.get("reason"),
self.continuations.pop(), and the ZapcodeError::RuntimeError(...) return are
formatted correctly and pass CI.

In `@crates/zapcode-core/tests/async_await.rs`:
- Around line 1010-1016: The failing rustfmt is due to the single-line assertion
formatting around the object check (the
assert!(map.get("__promise__").is_none(), "user object should not have
__promise__"); in the test matching arm for Value::Object); run cargo fmt or
reformat that assertion to the crate's expected multi-line style (split long
assertion and message across lines) so the tests compile with rustfmt, then
commit the formatted change.

---

Duplicate comments:
In `@crates/zapcode-core/src/vm/mod.rs`:
- Line 425: The code currently masks stack underflow by using let
callback_result = self.pop().unwrap_or(Value::Undefined);; change this to
propagate an error instead: replace unwrap_or with .ok_or(...) or
.ok_or_else(...) to return a VM error (e.g. VMError::StackUnderflow) and use the
? operator so the surrounding function returns Result; specifically update the
assignment of callback_result to something like let callback_result =
self.pop().ok_or(VMError::StackUnderflow)?; and ensure the enclosing function
signature and error type support returning that error.

---

Nitpick comments:
In `@crates/zapcode-core/src/vm/mod.rs`:
- Around line 658-665: Before pushing the new Continuation::ArrayMap onto
self.continuations, add a guard that checks the current continuation depth
against the VM's configured resource limit (e.g. a max_continuations or
continuation_depth limit on self or self.resource_limits); if the limit is
reached return an appropriate error/trap (same error type used elsewhere for
resource limits) instead of pushing, and only push after incrementing/validating
the count. Apply this check at the push site where Continuation::ArrayMap {
callback, source: arr, ... } is created so it consistently enforces continuation
depth like other VM resource limits.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a996a37d-b7f3-4e61-9e3b-af064acf3c8a

📥 Commits

Reviewing files that changed from the base of the PR and between e245133 and 443e163.

📒 Files selected for processing (2)
  • crates/zapcode-core/src/vm/mod.rs
  • crates/zapcode-core/tests/async_await.rs

@TheUncharted TheUncharted merged commit c4afcf0 into master Mar 13, 2026
20 checks passed
@TheUncharted TheUncharted changed the title core: add single-threaded event loop for Promise.all and async array callbacks feat: add single-threaded event loop for Promise.all and async array callbacks Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant