fix: add stack frame accounting to manifest builtins#866
fix: add stack frame accounting to manifest builtins#866johnbartholomew merged 3 commits intogoogle:masterfrom
Conversation
|
Could you investigate the test failures? ( |
004e62f to
a6c8b3e
Compare
|
I've rewrote the fix. Instead of using i.newCall() at every node (which ate up the global call stack on normal objects), I switched to a simple depth counter passed as parameter:
|
johnbartholomew
left a comment
There was a problem hiding this comment.
Thanks for the contribution!
This is good, but I don't like having the max depth fixed like this, as occasionally people have very strange use-cases. Although object nesting depth limit and call stack limit aren't quite the same thing they're close enough that I suggest applying the existing MaxStack limit as the depth cap here.
If you make depth count down and check against 0 instead of counting up and checking against a maxManifestDepth value, then you don't need to pass any extra limit argument around and the code stays mostly the same, you can just make the initial call with a depth of MaxStack, which you can get from the interpreter object (interpreter.stack.limit), though you might want to add an internal helper method to get that value cleanly.
Reusing the existing configurable limit will have the same default, but if someone is doing something strange they can set a much larger MaxStack (--max-stack for CLI users).
|
I think you are right, reusing MaxStack makes more sense. |
builtinManifestJSONEx, builtinManifestYamlDoc and builtinManifestTomlEx
use native Go recursion that bypasses MaxStack. A self-referential
object like {a: $} causes unbounded memory allocation until OOM.
Add i.newCall() at the top of each recursive closure, matching the
pattern used by manifestJSON in interpreter.go.
99bd6d1 to
456331f
Compare
|
Looks good! Merged :-) |
builtinManifestJSONEx,builtinManifestYamlDoc, andbuiltinManifestTomlExinbuiltins.goserialize values using native Go recursion (auxclosures ortomlRenderValue). These recursive functions don't calli.newCall(), so the interpreter'sMaxStackcounter never increments.A self-referential object like
{a: $}passed tostd.manifestJsonExcauses unbounded recursion and memory allocation (~900MB/s) until OOM, even whenMaxStackis set. The default manifestation path correctly catches this viamanifestJSONininterpreter.gowhich usesi.newCall().The fix adds
i.newCall(environment{}, false)and a deferredi.stack.popIfExists(stackSize)at the top of each recursive function, matching the pattern used bymanifestJSON.