Skip to content

fix(function): preserve source text for toString#375

Merged
dowdiness merged 4 commits into
mainfrom
issue-310-function-tostring-source
Jun 18, 2026
Merged

fix(function): preserve source text for toString#375
dowdiness merged 4 commits into
mainfrom
issue-310-function-tostring-source

Conversation

@dowdiness

@dowdiness dowdiness commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Preserve parser source spans for function/class productions and thread source text through AST/runtime function representations.
  • Return preserved source from Function.prototype.toString for parsed user functions/classes where available, while keeping native fallback handling.
  • Route dynamic Function/GeneratorFunction/AsyncFunction constructors through synthesized spec-shaped source parsing/storage, including the newline before ) {.
  • Address review findings for callable proxy native fallback, prefixed object/class method source spans, source-optional parse_tokens, hoisted generator/async declaration source-text threading, and source-derived token end offsets for regex/template/escaped identifier captures.

Refs #310.

Validation

  • moon check
  • moon test — 2106 passed, 0 failed
  • moon fmt && moon check
  • moon info
  • moon test parser --filter '*parse_tokens*'
  • moon test interpreter --filter '*Function.prototype.toString*'
  • NEW_MOON_MOD=0 make test262-filter FILTER=Function/prototype/toString
    • strict mode: 55 passed / 66 executed; 55 passed / 80 discovered; 14 skipped
    • non-strict mode: 55 passed / 66 executed; 55 passed / 80 discovered; 14 skipped
    • aggregate runner output: 110 passed / 132 executed, 110 passed / 160 discovered; 28 skipped; 22 failed

Notes

This reduces the Function/prototype/toString filter from the previous 108 aggregate failures to 22 aggregate failures. Remaining failures are mostly NativeFunction matcher/native formatting gaps for bound/proxy/symbol-named builtins.

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@dowdiness, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 57 minutes and 49 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0dcb9c08-12dc-469c-a295-bebda8ce7645

📥 Commits

Reviewing files that changed from the base of the PR and between 001ef3d and e63a87e.

📒 Files selected for processing (29)
  • ast/ast.mbt
  • ast/pkg.generated.mbti
  • compiler/bytecode_lower.mbt
  • compiler/closure_conversion.mbt
  • docs/supported-features.md
  • interpreter/interpreter_test.mbt
  • interpreter/runtime/async.mbt
  • interpreter/runtime/class.mbt
  • interpreter/runtime/construct.mbt
  • interpreter/runtime/conversions.mbt
  • interpreter/runtime/eval_contains.mbt
  • interpreter/runtime/eval_expr.mbt
  • interpreter/runtime/exec_stmt.mbt
  • interpreter/runtime/generator.mbt
  • interpreter/runtime/hoisting.mbt
  • interpreter/runtime/interpreter.mbt
  • interpreter/runtime/module_default_export.mbt
  • interpreter/runtime/module_graph.mbt
  • interpreter/runtime/modules.mbt
  • interpreter/runtime/pkg.generated.mbti
  • interpreter/runtime/property.mbt
  • interpreter/runtime/value.mbt
  • interpreter/stdlib/builtins_function.mbt
  • parser/expr.mbt
  • parser/parser.mbt
  • parser/parser_test.mbt
  • parser/pkg.generated.mbti
  • parser/stmt.mbt
  • static_semantics/declarations.mbt
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-310-function-tostring-source

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3785541a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread parser/parser.mbt
let parser = Parser::new(tokens)
pub fn parse_tokens(
tokens : Array[@token.Token],
source? : String = "",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not slice an empty source in parse_tokens

parse_tokens still defaults source to "", but the new function/class AST paths call consume_source_text, which slices self.source using token offsets. Existing callers that only pass tokens, including the benchmark parse paths that call @parser.parse_tokens(tokens), will hit this for any tokenized program containing functions/classes and either abort on the out-of-range substring or attach bogus source text. Please require/provide the original source or make source capture optional when no source was supplied.

Useful? React with 👍 / 👎.

Comment thread parser/expr.mbt Outdated
Comment thread parser/expr.mbt Outdated
Comment thread interpreter/runtime/hoisting.mbt Outdated
@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Run: https://github.com/dowdiness/js_engine/actions/runs/27741321336

startup/tiny_program is the PR #153 / issue #141 guardrail for built-in realm-stamping startup cost.

Stage summary

stage benchmarks total mean slowest benchmark slowest mean noisy rows
startup 3 2.617 ms startup/tiny_program 1.320 ms 0
frontend 7 0.843 ms pipeline/parse_heavy 0.478 ms 2
execution 26 15296.533 ms exec/fibonacci_30 14010.526 ms 2

Focused bytecode base-vs-head comparison

Base-vs-head deltas are reporting-only. Negative delta and PR/base < 1.00x mean the PR is faster; interpret high-CV or noisy rows cautiously.

benchmark stage base mean PR mean delta PR/base base CV PR CV noisy
baseline/bytecode/closure_factory execution 15.168 ms 14.571 ms -3.9% 0.96x 6.4% 5.6% no
pipeline/bytecode/evaluate execution 9.520 ms 8.772 ms -7.9% 0.92x 4.0% 1.2% no
isolate/bytecode/call_frame execution 7.810 ms 7.531 ms -3.6% 0.96x 3.3% 1.6% no
isolate/bytecode/runtime_helpers execution 11.301 ms 11.017 ms -2.5% 0.97x 1.0% 2.6% no
isolate/bytecode/local_access execution 38.228 ms 37.898 ms -0.9% 0.99x 1.7% 1.0% no
isolate/bytecode/env_access execution 38.320 ms 37.554 ms -2.0% 0.98x 1.2% 1.1% no
isolate/bytecode/captured_access execution 38.678 ms 38.237 ms -1.1% 0.99x 1.6% 2.7% no
isolate/bytecode/dispatch_stack execution 23.978 ms 24.358 ms +1.6% 1.02x 2.3% 2.6% no

Base-vs-head comparison

benchmark stage base mean PR mean delta PR/base base CV PR CV noisy
startup/tiny_program startup 1.354 ms 1.320 ms -2.5% 0.97x 6.9% 5.7% no
lexer/small frontend 0.030 ms 0.031 ms +3.0% 1.03x 24.9% 27.8% base, PR
lexer/large frontend 0.256 ms 0.256 ms +0.3% 1.00x 0.6% 0.9% no
exec/fibonacci_30 execution 14528.857 ms 14010.526 ms -3.6% 0.96x 0.8% 0.6% no
exec/property_chain execution 13.822 ms 13.460 ms -2.6% 0.97x 4.8% 8.7% no
startup/phase/parse_tiny frontend 0.002 ms 0.002 ms +5.7% 1.06x 2.1% 0.9% no
startup/phase/new_interpreter startup 1.272 ms 1.297 ms +2.0% 1.02x 9.6% 11.9% no
startup/phase/execute_preparsed_tiny execution 0.001 ms 0.000 ms -4.1% 0.96x 0.8% 1.2% no
startup/phase/event_loop_drain_empty startup 0.000 ms 0.000 ms +2.2% 1.02x 1.1% 1.1% no
startup/phase/result_stringify_output execution 0.000 ms 0.000 ms +0.0% 1.00x 0.5% 1.2% no
exec/array_map_filter execution 20.711 ms 20.894 ms +0.9% 1.01x 15.9% 16.2% base, PR
exec/closure_factory execution 30.657 ms 30.033 ms -2.0% 0.98x 6.0% 5.7% no
baseline/closure_legacy/closure_factory execution 31.036 ms 28.864 ms -7.0% 0.93x 9.2% 8.4% no
baseline/bytecode/closure_factory execution 15.168 ms 14.571 ms -3.9% 0.96x 6.4% 5.6% no
isolate/bytecode/dispatch_stack execution 23.978 ms 24.358 ms +1.6% 1.02x 2.3% 2.6% no
isolate/bytecode/local_access execution 38.228 ms 37.898 ms -0.9% 0.99x 1.7% 1.0% no
isolate/bytecode/env_access execution 38.320 ms 37.554 ms -2.0% 0.98x 1.2% 1.1% no
isolate/bytecode/captured_access execution 38.678 ms 38.237 ms -1.1% 0.99x 1.6% 2.7% no
isolate/bytecode/call_frame execution 7.810 ms 7.531 ms -3.6% 0.96x 3.3% 1.6% no
isolate/bytecode/runtime_helpers execution 11.301 ms 11.017 ms -2.5% 0.97x 1.0% 2.6% no
isolate/bytecode/property_get execution 43.976 ms 45.534 ms +3.5% 1.04x 1.3% 2.5% no
isolate/bytecode/property_set execution 39.300 ms 39.883 ms +1.5% 1.01x 1.1% 4.0% no
isolate/bytecode/method_call execution 8.592 ms 8.357 ms -2.7% 0.97x 1.0% 1.6% no
isolate/bytecode/object_literal execution 14.092 ms 13.914 ms -1.3% 0.99x 0.9% 9.6% no
isolate/bytecode/array_literal execution 15.364 ms 14.637 ms -4.7% 0.95x 1.5% 1.5% no
exec/for_of execution 5.788 ms 5.694 ms -1.6% 0.98x 3.4% 2.5% no
exec/arithmetic_loop execution 941.385 ms 825.111 ms -12.4% 0.88x 2.3% 1.9% no
exec/object_construction execution 7.718 ms 7.163 ms -7.2% 0.93x 5.5% 5.4% no
exec/string_ops execution 2.085 ms 2.187 ms +4.9% 1.05x 23.2% 25.4% base, PR
pipeline/exec/lex frontend 0.027 ms 0.027 ms -1.3% 0.99x 0.5% 2.7% no
pipeline/exec/parse frontend 0.027 ms 0.027 ms -2.7% 0.97x 8.6% 4.2% no
pipeline/exec/evaluate execution 28.894 ms 25.497 ms -11.8% 0.88x 11.7% 14.7% no
pipeline/closure_legacy/evaluate execution 26.887 ms 24.841 ms -7.6% 0.92x 3.8% 4.5% no
pipeline/bytecode/compile frontend 0.022 ms 0.022 ms +1.0% 1.01x 27.1% 29.2% base, PR
pipeline/bytecode/evaluate execution 9.520 ms 8.772 ms -7.9% 0.92x 4.0% 1.2% no
pipeline/parse_heavy frontend 0.496 ms 0.478 ms -3.6% 0.96x 6.5% 5.7% no

Mean-time chart (log scale)

benchmark stage mean chart
startup/tiny_program startup 1.320 ms ##
lexer/small frontend 0.031 ms ⚠ #
lexer/large frontend 0.256 ms #
exec/fibonacci_30 execution 14010.526 ms ##############################
exec/property_chain execution 13.460 ms ########
startup/phase/parse_tiny frontend 0.002 ms #
startup/phase/new_interpreter startup 1.297 ms ##
startup/phase/execute_preparsed_tiny execution 0.000 ms #
startup/phase/event_loop_drain_empty startup 0.000 ms #
startup/phase/result_stringify_output execution 0.000 ms #
exec/array_map_filter execution 20.894 ms ⚠ #########
exec/closure_factory execution 30.033 ms ##########
baseline/closure_legacy/closure_factory execution 28.864 ms ##########
baseline/bytecode/closure_factory execution 14.571 ms ########
isolate/bytecode/dispatch_stack execution 24.358 ms ##########
isolate/bytecode/local_access execution 37.898 ms ###########
isolate/bytecode/env_access execution 37.554 ms ###########
isolate/bytecode/captured_access execution 38.237 ms ###########
isolate/bytecode/call_frame execution 7.531 ms ######
isolate/bytecode/runtime_helpers execution 11.017 ms #######
isolate/bytecode/property_get execution 45.534 ms ############
isolate/bytecode/property_set execution 39.883 ms ###########
isolate/bytecode/method_call execution 8.357 ms #######
isolate/bytecode/object_literal execution 13.914 ms ########
isolate/bytecode/array_literal execution 14.637 ms ########
exec/for_of execution 5.694 ms #####
exec/arithmetic_loop execution 825.111 ms #####################
exec/object_construction execution 7.163 ms ######
exec/string_ops execution 2.187 ms ⚠ ###
pipeline/exec/lex frontend 0.027 ms #
pipeline/exec/parse frontend 0.027 ms #
pipeline/exec/evaluate execution 25.497 ms ##########
pipeline/closure_legacy/evaluate execution 24.841 ms ##########
pipeline/bytecode/compile frontend 0.022 ms ⚠ #
pipeline/bytecode/evaluate execution 8.772 ms #######
pipeline/parse_heavy frontend 0.478 ms #

Closure-conversion comparison

  • unavailable

@dowdiness dowdiness force-pushed the issue-310-function-tostring-source branch from e378554 to 58fc5de Compare June 18, 2026 06:09

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c2731f8f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread parser/parser.mbt Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e63a87e872

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

compile_ternary(cond, then_expr, else_expr)
FuncExpr(name, params, body, _) => compile_func_expr(name, params, body)
FuncExprExt(name, params, rest_param, body, _) =>
FuncExpr(name, params, body, _, _) => compile_func_expr(name, params, body)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve source text in compiled functions

When code runs through the public run_compiled / --closure-conversion path, the parser has already attached the new source_text field, but this branch discards it before building the compiled function. The compiled function is later represented as an InterpreterCallableWithContext with no [[SourceText]], so inputs like run_compiled("console.log((function f(){}).toString())") still print a [native code] fallback instead of the parsed source, unlike the normal interpreter path. Thread the source text through the compiled function/arrow/declaration helpers so the two execution modes stay aligned.

Useful? React with 👍 / 👎.

@dowdiness dowdiness merged commit 5a5db69 into main Jun 18, 2026
15 checks passed
@dowdiness dowdiness deleted the issue-310-function-tostring-source branch June 18, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant