vm: avoid stack overflow on recursive accessor calls

[Flamki]

This Pull Request fixes/closes #4535.

Summary

• Track nested host-driven VM re-entry (JsObject::call / JsObject::construct) with a new Vm::host_call_depth counter.
• Include host_call_depth in Context::check_runtime_limits() recursion checks so recursive accessor calls fail with RuntimeLimitError::Recursion instead of overflowing the native stack.
• Unignore and strengthen the regression test in �m/tests.rs by exercising the async-generator hen getter recursion path with a higher recursion limit.

Notes

• I could not run cargo in this environment (toolchain not available), so CI is expected to validate formatting, lint, and tests.

[github-actions]

Test262 conformance changes

Test result	main count	PR count	difference
Total	52,862	52,862	0
Passed	49,504	49,504	0
Ignored	2,262	2,262	0
Failed	1,096	1,096	0
Panics	0	0	0
Conformance	93.65%	93.65%	0.00%

[Flamki]

Follow-up hardening pushed in 9966682.

What I changed:

• Rebased branch on latest main.

• Added a dedicated regression test for recursive setters:

ecursion_in_setter_throws_uncatchable_error

• asserts RuntimeLimitError::Recursion for set x(v) { this.x = v; } recursion.

This complements the existing async-generator getter regression and ensures both accessor call directions are covered by runtime-limit behavior.

[jedel1043]

I'd suggest measuring the performance impact of this change, because keeping track of recursive native calls sounds like it could be detrimental for perf. You can use https://github.com/boa-dev/data/blob/main/bench/bench-v8/combined.js for this.

[Flamki]

@jedel1043 thanks for raising this, I took a careful pass on perf.

I built clean release binaries for:

• upstream/main (549d14a5)
• this branch (390f0a1b, plus the latest test-only commit)

I first tried to run the exact benchmark you suggested from boa-dev/data (bench/bench-v8/combined.js). On my Windows setup, that script stack-overflows on both revisions before producing comparable scores, so I couldn’t get a reliable canonical A/B from it locally.

To still get signal, I ran fallback release A/B hot-path measurements (11 runs each, first run dropped as warmup) focused on accessor/property/function throughput:

Script	main mean (ms)	branch mean (ms)	delta
accessor_hot.js	3894.162	3629.003	-6.81%
plain_prop_hot.js	1446.026	1381.159	-4.49%
func_hot.js	7877.123	7616.737	-3.31%

From this local fallback measurement, I did not observe a regression from host_call_depth tracking.

I also lowered both recursion regression test limits to 128 for platform stability and re-ran:
cargo test -p boa_engine vm::tests -> 26 passed, 0 failed.

[jedel1043]

Thanks!

Maybe in the future we'll finally have coroutines to lift all the inner calls to the main evaluation loop, but in the meantime we need to work around this limitation.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.12%. Comparing base (6ddc2b4) to head (8b867e2).
⚠️ Report is 694 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4699      +/-   ##
==========================================
+ Coverage   47.24%   57.12%   +9.88%     
==========================================
  Files         476      549      +73     
  Lines       46892    60406   +13514     
==========================================
+ Hits        22154    34507   +12353     
- Misses      24738    25899    +1161     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.