Skip to content

interp: block return must not ride the invokeEx code0 fast path - #3572

Merged
borisbat merged 2 commits into
masterfrom
bbatkin/fix-block-return-leak
Jul 26, 2026
Merged

interp: block return must not ride the invokeEx code0 fast path#3572
borisbat merged 2 commits into
masterfrom
bbatkin/fix-block-return-leak

Conversation

@borisbat

@borisbat borisbat commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

interp: block return must not ride the invokeEx code0 fast path

The bug

Context::invokeEx — the batch block-invocation primitive behind the ast enumerators (for_each_function, for_each_generic, …) — has a fast path: when the closure body is a void single statement with no finally (code0), it hands the enumeration lambda the bare statement instead of the SimNode_ClosureBlock. But the ClosureBlock's eval is what clears stopForReturn after the body runs. With the fast path, a firing bare return inside such a closure (return-from-block, the everyday "skip this item" idiom):

  1. leaves stopForReturn set, so every remaining iteration of the enumeration no-ops, and
  2. leaks the flag into the calling function, silently skipping everything after the enumeration call (damage stops at the next function-call boundary, which is why nothing ever crashed).

Interpreter only — JIT and AOT were always correct, and per-item Context::invoke (the dir() pattern) was safe because it evals block.body.

The longest-lived visible casualty: rst.das documents() — its "Uncategorized" loop is exactly this shape (auto-inline canonicalizes if (cond) { return }; rest into a single if/else statement, putting the closure on the fast path). Every function not matched by a group_by_regex was silently dropped from the generated docs, and the docs/uncategorized preflight gate passed precisely because the loop was dead.

The fix

ast_simulate.cpp: C0 additionally requires !expr->hasReturn. Return-bearing single-statement closures take the full ClosureBlock path; the fast path stays for the common no-return case.

Test

tests/ast/test_closure_block_return.das — failing-test-first: a single-statement closure (fast path) and a multi-statement closure (never on the fast path) run the same enumeration with a firing return and must agree, plus a tail sentinel proving the flag does not leak past the enumeration. Red before the fix, green after.

Suites on the fixed binary: language 1443/1443, ast, type_lattice, spirv, metal — all green. Full preflight: the entire heavy matrix (tests-interp / tests-jit / tests-aot) passed first run.

Doc fallout (second commit)

With the Uncategorized loop alive again, the functions it had been silently dropping surface in the generated docs. Grouped + documented so the doc gates stay green:

  • fio: set_env_variable (joins the env getters; new handmade entry), long_fread/long_fwrite (join File manipulation)
  • logger: logger_init_tee, logger_set_tee (join Setup / Output control; //! docs already existed)
  • jobque: set_jobque_thread_team_mode / get_jobque_thread_team_mode / set_current_thread_priority — hidden with the rest of the tuning-knob family, new handmade entries

Review note: an earlier revision of this branch accidentally glob-deleted and re-derived the existing jobque handmade descriptions; Copilot's review caught the rewrites diverging from the C++ semantics. The curated files are restored verbatim from master — only the four genuinely new entries above are added.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q8bZnWkVWeXy8jfrxSojso

Context::invokeEx hands enumerators (for_each_function et al.) the closure
body's single statement bare when code0 is set — skipping the
SimNode_ClosureBlock epilogue that clears stopForReturn. A firing bare
`return` inside such a block then killed the remaining iterations AND
leaked the stop flag into the calling function, silently skipping its
tail. Longest-lived visible casualty: rst.das documents() dropped every
Uncategorized function from the generated docs, and gate 3 passed
precisely because the loop was dead. JIT and AOT were always correct.

Fix: C0 additionally requires !expr->hasReturn, so return-bearing
single-statement closures take the full ClosureBlock path. Failing test
first: tests/ast/test_closure_block_return.das (red pre-fix, green now).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q8bZnWkVWeXy8jfrxSojso
Copilot AI review requested due to automatic review settings July 26, 2026 09:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes an interpreter-only control-flow bug where return inside certain single-statement closure blocks (invoked via Context::invokeEx fast path) could leave stopForReturn set, causing subsequent enumeration iterations and caller code to be silently skipped. The PR also adds a regression test and updates stdlib documentation groupings/handmade entries impacted by the corrected interpreter behavior.

Changes:

  • Tighten the interpreter “code0” closure fast path to exclude return-bearing closure blocks (!expr->hasReturn), ensuring the SimNode_ClosureBlock epilogue clears stopForReturn.
  • Add an AST regression test that reproduces the fast-path case and verifies the stop flag doesn’t leak past enumeration.
  • Update das2rst grouping regexes and refresh/expand several handmade stdlib RST blurbs surfaced by the fix.

Reviewed changes

Copilot reviewed 84 out of 84 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/ast/test_closure_block_return.das Adds regression test covering block-return behavior in invokeEx enumerators.
src/ast/ast_simulate.cpp Prevents code0 fast path for closures that contain a return.
doc/reflections/das2rst.das Updates doc grouping regexes for fio/jobque/logger.
doc/source/stdlib/handmade/function-jobque-with_stream-0xc332973ad62f305b.rst Updates handmade blurb for with_stream.
doc/source/stdlib/handmade/function-jobque-with_lock_box-0x34eec090d1eadaed.rst Updates handmade blurb for with_lock_box.
doc/source/stdlib/handmade/function-jobque-with_job_status-0xc2b37ac2efeac9c9.rst Updates handmade blurb for with_job_status.
doc/source/stdlib/handmade/function-jobque-with_job_que-0x47bb68b710114bf6.rst Updates handmade blurb for with_job_que.
doc/source/stdlib/handmade/function-jobque-with_channel-0x6445159c1f55b7b4.rst Updates handmade blurb for with_channel.
doc/source/stdlib/handmade/function-jobque-with_atomic64-0xfd2f95909eeb9890.rst Updates handmade blurb for with_atomic64.
doc/source/stdlib/handmade/function-jobque-with_atomic32-0x8516d8ba5aee7451.rst Updates handmade blurb for with_atomic32.
doc/source/stdlib/handmade/function-jobque-team_parallel_stages_invoke-0xb10de4a3b1ff3b43.rst Shortens internal-invocation description.
doc/source/stdlib/handmade/function-jobque-team_parallel_for_invoke-0xf4e98d4b1066a5d0.rst Shortens internal-invocation description.
doc/source/stdlib/handmade/function-jobque-team_parallel_for_indexed_invoke-0x5fe878666ef9a86b.rst Shortens internal-invocation description.
doc/source/stdlib/handmade/function-jobque-stream_remove-0xb53c58ce1a8ff30f.rst Updates handmade blurb for stream_remove.
doc/source/stdlib/handmade/function-jobque-stream_create-0x87e88f05bb49ad9a.rst Updates handmade blurb for stream_create.
doc/source/stdlib/handmade/function-jobque-set-0xab92b04192520111.rst Updates handmade blurb for set.
doc/source/stdlib/handmade/function-jobque-set_jobque_worker_spin-0xb5dd96322b329b37.rst Updates handmade blurb for worker spin setting.
doc/source/stdlib/handmade/function-jobque-set_jobque_worker_limit-0x12c022c49e013ea4.rst Updates handmade blurb for worker limit setting.
doc/source/stdlib/handmade/function-jobque-set_jobque_threads_cap-0x9e136f3eb0689da6.rst Updates handmade blurb for threads cap setting.
doc/source/stdlib/handmade/function-jobque-set_jobque_thread_team_mode-0x3ea171573d71f2e0.rst Adds handmade blurb for per-thread team-mode override.
doc/source/stdlib/handmade/function-jobque-set_jobque_team_rank_gate-0xb605728bf3dffc66.rst Updates handmade blurb for rank gate setting.
doc/source/stdlib/handmade/function-jobque-set_jobque_team_prof-0xf24c498ed3d3b4f5.rst Updates handmade blurb for team profiling enable.
doc/source/stdlib/handmade/function-jobque-set_jobque_team_mode-0xa045c3547b8dcf99.rst Updates handmade blurb for team mode.
doc/source/stdlib/handmade/function-jobque-set_jobque_join_spin-0x9c8fe6474de5b554.rst Updates handmade blurb for join spin setting.
doc/source/stdlib/handmade/function-jobque-set_jobque_fork_skip_heap_reset-0xd78b4b00651279ac.rst Updates handmade blurb for fork heap reset control.
doc/source/stdlib/handmade/function-jobque-set_jobque_fork_pool-0x39e2081080641e22.rst Updates handmade blurb for fork pool control.
doc/source/stdlib/handmade/function-jobque-set_jobque_batch_dispatch-0x49f1130dd2a75049.rst Updates handmade blurb for batch dispatch mode.
doc/source/stdlib/handmade/function-jobque-set_jobque_affinity-0x3da70f47809a52e5.rst Updates handmade blurb for affinity setting.
doc/source/stdlib/handmade/function-jobque-set_current_thread_priority-0x30274d0f647e878a.rst Adds handmade blurb for setting calling thread priority.
doc/source/stdlib/handmade/function-jobque-reset_jobque_team_prof-0x9be099e0c87e3e82.rst Updates handmade blurb for team profiling reset.
doc/source/stdlib/handmade/function-jobque-release-0x6c65fbac1ec7c6c9.rst Updates handmade blurb for release.
doc/source/stdlib/handmade/function-jobque-notify-0x9a35438e53d274c7.rst Updates handmade blurb for notify.
doc/source/stdlib/handmade/function-jobque-notify_and_release-0xd2b5630a04957e47.rst Updates handmade blurb for notify_and_release.
doc/source/stdlib/handmade/function-jobque-new_thread_invoke-0x694f3744ccc32b35.rst Updates internal trampoline description.
doc/source/stdlib/handmade/function-jobque-new_job_invoke-0x54460a03fb4b6620.rst Updates internal trampoline description.
doc/source/stdlib/handmade/function-jobque-new_debugger_thread-0x6d43cc92315fb5f9.rst Updates internal trampoline description.
doc/source/stdlib/handmade/function-jobque-lock_box_remove-0xc3fb388195334355.rst Updates handmade blurb for lock_box_remove.
doc/source/stdlib/handmade/function-jobque-lock_box_create-0x5b995e9384235055.rst Updates handmade blurb for lock_box_create.
doc/source/stdlib/handmade/function-jobque-join-0xf9ef0386c6ba2df6.rst Updates handmade blurb for join.
doc/source/stdlib/handmade/function-jobque-jobque_try_run_one-0x30a7e96889b38133.rst Updates handmade blurb for jobque_try_run_one.
doc/source/stdlib/handmade/function-jobque-jobque_trace_tag-0x6ebad8ff0ee8c71b.rst Updates handmade blurb for trace tagging.
doc/source/stdlib/handmade/function-jobque-jobque_trace_stop-0x604f59d3232be1b7.rst Updates handmade blurb for stopping trace capture.
doc/source/stdlib/handmade/function-jobque-jobque_trace_start-0x46ca8435a72075aa.rst Updates handmade blurb for starting trace capture.
doc/source/stdlib/handmade/function-jobque-jobque_trace_save-0xda608a2b0f094967.rst Updates handmade blurb for saving traces.
doc/source/stdlib/handmade/function-jobque-jobque_trace_marker-0x23d3c8a055ea0170.rst Updates handmade blurb for trace markers.
doc/source/stdlib/handmade/function-jobque-jobque_trace_marker_name-0x11d82ed54aded050.rst Updates handmade blurb for marker-name registration.
doc/source/stdlib/handmade/function-jobque-jobque_trace_category-0x9e6a5d0ed05ba6da.rst Updates handmade blurb for trace categories.
doc/source/stdlib/handmade/function-jobque-job_status_remove-0xcb87a6cccf8720da.rst Updates handmade blurb for job_status_remove.
doc/source/stdlib/handmade/function-jobque-job_status_create-0xfd9f81d0a23b230b.rst Updates handmade blurb for job_status_create.
doc/source/stdlib/handmade/function-jobque-is_job_que_shutting_down-0x8ebb7c0bf4ea7622.rst Updates handmade blurb for shutdown query.
doc/source/stdlib/handmade/function-jobque-is_job_que_available-0x991030b0da8905df.rst Updates handmade blurb for availability query.
doc/source/stdlib/handmade/function-jobque-inc-0x9385ca51819c5a80.rst Updates handmade blurb for inc.
doc/source/stdlib/handmade/function-jobque-get-0x69c8ac06b69bcfc3.rst Updates handmade blurb for get.
doc/source/stdlib/handmade/function-jobque-get_total_hw_threads-0xbf94763f377d7f4.rst Updates handmade blurb for thread count query.
doc/source/stdlib/handmade/function-jobque-get_total_hw_jobs-0x523917b03cee272d.rst Updates handmade blurb for default worker count query.
doc/source/stdlib/handmade/function-jobque-get_total_hw_cores-0x8afaffaf0f25be77.rst Updates handmade blurb for core count query.
doc/source/stdlib/handmade/function-jobque-get_jobque_worker_limit-0xef4a5a811433dc0b.rst Updates handmade blurb for worker limit query.
doc/source/stdlib/handmade/function-jobque-get_jobque_thread_team_mode-0xadb77f2665209111.rst Adds handmade blurb for per-thread team-mode query.
doc/source/stdlib/handmade/function-jobque-get_jobque_team_rank_gate-0x7078181c65dc10bb.rst Updates handmade blurb for rank gate query.
doc/source/stdlib/handmade/function-jobque-get_jobque_team_prof-0xa0a1388650f3262e.rst Updates handmade blurb for profiling query.
doc/source/stdlib/handmade/function-jobque-get_jobque_team_prof_react-0xf94ead1fbe5f7351.rst Updates handmade blurb for profiling react-latency query.
doc/source/stdlib/handmade/function-jobque-get_jobque_team_prof_counts-0xa6bb9f9f222b77ae.rst Updates handmade blurb for profiling counters query.
doc/source/stdlib/handmade/function-jobque-get_jobque_team_mode-0x85011aef60133a02.rst Updates handmade blurb for team mode query.
doc/source/stdlib/handmade/function-jobque-get_jobque_affinity-0xe5f0154608db08b5.rst Updates handmade blurb for affinity query.
doc/source/stdlib/handmade/function-jobque-flush_jobque_batch-0xd7c42ce625ed2b3f.rst Updates handmade blurb for flushing batch dispatch.
doc/source/stdlib/handmade/function-jobque-destroy_job_que-0x2de74c871d878a97.rst Updates handmade blurb for destroying job queue.
doc/source/stdlib/handmade/function-jobque-dec-0xdad0e2b03a400865.rst Updates handmade blurb for dec.
doc/source/stdlib/handmade/function-jobque-create_job_que-0x9b17cb32d334c8f2.rst Updates handmade blurb for creating job queue.
doc/source/stdlib/handmade/function-jobque-count_jobque_leaks-0xb4ae32ea24efdbdf.rst Updates handmade blurb for leak counter.
doc/source/stdlib/handmade/function-jobque-channel_remove-0x5ed45a6930f1af86.rst Updates handmade blurb for channel_remove.
doc/source/stdlib/handmade/function-jobque-channel_create-0x6acba3a15dc72bae.rst Updates handmade blurb for channel_create.
doc/source/stdlib/handmade/function-jobque-atomic64_remove-0x1402c05879740ab2.rst Updates handmade blurb for atomic64_remove.
doc/source/stdlib/handmade/function-jobque-atomic64_create-0xe6ee4a8caab11689.rst Updates handmade blurb for atomic64_create.
doc/source/stdlib/handmade/function-jobque-atomic32_remove-0x61c1daf7b3c7f6df.rst Updates handmade blurb for atomic32_remove.
doc/source/stdlib/handmade/function-jobque-atomic32_create-0xa059c9a589504660.rst Updates handmade blurb for atomic32_create.
doc/source/stdlib/handmade/function-jobque-append-0xec17dd041d13b3e4.rst Updates handmade blurb for append.
doc/source/stdlib/handmade/function-jobque-add_ref-0xbcb50b340fbd41ed.rst Updates handmade blurb for add_ref.
doc/source/stdlib/handmade/function-jobque-_dot__rq_total-0x29c01675b6c8c5c7.rst Updates handmade blurb for .total.
doc/source/stdlib/handmade/function-jobque-_dot__rq_size-0xc2f590b3bd8833d4.rst Updates handmade blurb for .size.
doc/source/stdlib/handmade/function-jobque-_dot__rq_isValid-0xab05ad1d845c8972.rst Updates handmade blurb for .isValid.
doc/source/stdlib/handmade/function-jobque-_dot__rq_isReady-0xe1ef2208f6d0b04c.rst Updates handmade blurb for .isReady.
doc/source/stdlib/handmade/function-jobque-_dot__rq_isEmpty-0xa29c0d2fe06d4474.rst Updates handmade blurb for .isEmpty.
doc/source/stdlib/handmade/function-fio-set_env_variable-0xf7b1b4df69de309.rst Adds handmade blurb for set_env_variable.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread doc/source/stdlib/handmade/function-jobque-channel_create-0x6acba3a15dc72bae.rst Outdated
Comment thread doc/source/stdlib/handmade/function-jobque-stream_create-0x87e88f05bb49ad9a.rst Outdated
Comment thread doc/source/stdlib/handmade/function-jobque-lock_box_create-0x5b995e9384235055.rst Outdated
Comment thread doc/source/stdlib/handmade/function-jobque-atomic32_create-0xa059c9a589504660.rst Outdated
Comment thread doc/source/stdlib/handmade/function-jobque-atomic64_create-0xe6ee4a8caab11689.rst Outdated
Copilot AI review requested due to automatic review settings July 26, 2026 09:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 84 out of 84 changed files in this pull request and generated 7 comments.

Comment thread doc/source/stdlib/handmade/function-jobque-with_stream-0xc332973ad62f305b.rst Outdated
The invokeEx block-return fix brings rst.das documents()'s Uncategorized
loop back to life, surfacing functions that had been silently dropped
from the generated docs. Groups + handmade entries for the new ones:
fio set_env_variable + long_fread/long_fwrite, logger_init_tee/
logger_set_tee, and the per-thread jobque dials (set/get_jobque_thread_
team_mode, set_current_thread_priority — hidden with the tuning-knob
family). Existing jobque handmade descriptions are untouched (an earlier
revision of this branch accidentally rewrote them; restored verbatim —
Copilot review caught the divergences).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q8bZnWkVWeXy8jfrxSojso
Copilot AI review requested due to automatic review settings July 26, 2026 10:31
@borisbat
borisbat force-pushed the bbatkin/fix-block-return-leak branch from b2687c8 to 60ef003 Compare July 26, 2026 10:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@borisbat
borisbat merged commit 4783583 into master Jul 26, 2026
38 checks passed
@borisbat
borisbat deleted the bbatkin/fix-block-return-leak branch July 26, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants