fix(module): drop AST and source text after module initialization#4855
Merged
hansl merged 3 commits intoboa-dev:mainfrom Mar 4, 2026
Merged
Conversation
added 2 commits
March 4, 2026 16:19
Free the parsed boa_ast::Module and SourceText from ModuleCode once initialize_environment() completes. Both fields are only needed during compilation; wrapping them in RefCell<Option<_>> and taking them at the start of that function lets the full parse tree be reclaimed immediately after a module is linked, reducing persistent memory use in workloads that load many modules. Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com>
hansl
requested changes
Mar 4, 2026
| let source = self | ||
| .code | ||
| .source | ||
| .borrow_mut() |
Contributor
There was a problem hiding this comment.
You should be able to just call take() on the RefCell here. It is implemented when default is implemented which is the case for an Option.
Author
|
Hello @hansl @jedel1043 |
hansl
approved these changes
Mar 4, 2026
Test262 conformance changes
Tested PR commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4855 +/- ##
==========================================
+ Coverage 47.24% 57.19% +9.94%
==========================================
Files 476 554 +78
Lines 46892 60473 +13581
==========================================
+ Hits 22154 34585 +12431
- Misses 24738 25888 +1150 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Member
|
This kinda makes it impossible to re-evaluate a module again. Might be fine I guess |
ashddev
pushed a commit
to ashddev/boa
that referenced
this pull request
Mar 4, 2026
…a-dev#4855) # Summary This PR fixes a memory retention issue in the module system where the **parsed AST (`boa_ast::Module`) and the module source text were kept in memory even after compilation finished**. These structures are only required during the **module initialization phase** (`initialize_environment`). However, they were previously stored as permanent fields inside `ModuleCode`, meaning every loaded module kept its entire AST and source text allocated for the lifetime of the process. In workloads that load many modules, this can lead to **steady memory growth over time**. --- # Where the Issue Occurred The problem originates in: ``` core/engine/src/module/source.rs ``` `ModuleCode` previously stored the full AST and source text directly: ```rust struct ModuleCode { has_tla: bool, requested_modules: IndexSet<ModuleRequest>, source: boa_ast::Module, source_text: SourceText, } ``` Both values are only used inside `SourceTextModule::initialize_environment()`, but they remained allocated afterward. Since modules are cached by `SimpleModuleLoader`, the AST and source text would stay in memory indefinitely. --- # Fix The AST and source text are now stored as optional values so they can be **consumed and dropped once compilation is complete**. ```rust source: RefCell<Option<boa_ast::Module>>, source_text: RefCell<Option<SourceText>>, ``` They are extracted during initialization: ```rust let source = self.code.source.borrow_mut().take() .expect("module source consumed before initialize_environment"); ``` After this point, the AST and source text are no longer retained by the module. --- # Impact This prevents modules from permanently holding large AST structures that are no longer needed after compilation. In module-heavy workloads (such as large dependency graphs or repeated module loading), this **keeps memory usage stable instead of growing with every loaded module**. --- # Result * AST memory is released immediately after module linking * Modules retain only the data needed for execution * Long-running Boa processes avoid unnecessary memory growth --------- Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Mar 6, 2026
## Context This PR extends the idea introduced in #4855. PR #4855 fixed a memory retention issue for **modules**, where the parsed AST and source text were kept in memory even after module initialization. This PR applies the same idea to **scripts**, ensuring that the parsed AST is released once script execution has completed. # Summary This PR prevents scripts from keeping their parsed AST after compilation. Right now `Script::Inner` stores the full AST in the `source` field: ```rust struct Inner { realm: Realm, source: boa_ast::Script, source_text: SourceText, codeblock: GcRefCell<Option<Gc<CodeBlock>>>, } ``` The AST is only needed while generating bytecode in `Script::codeblock()`. After compilation it is not used anymore, but it still stays in `Script::Inner`. Function objects created from the script keep a reference to the script (`OrdinaryFunction::script_or_module`). Because of this, the script can stay alive for a long time, which also keeps the whole AST in memory. # Steps to Reproduce In a long-running program (for example a REPL or server), repeatedly run scripts that define functions: ```rust let script = Script::parse(Source::from_bytes(src), None, &mut context)?; script.evaluate(&mut context)?; ``` If those functions are stored on the global object, they keep the script alive. Since the script holds the AST, memory usage keeps growing as more scripts run. # Impact This mostly affects long-running programs that embed Boa. Every script that defines functions can keep its AST in memory forever. AST structures are usually much bigger than the source code, so memory usage can grow over time. Modules already drop their AST after linking, but scripts did not. # Fix `Script::Inner::source` is changed to: ```rust source: RefCell<Option<boa_ast::Script>> ``` The AST is used during compilation and then removed once the `CodeBlock` is created: ```rust *self.inner.source.borrow_mut() = None; ``` # Result After this change, the AST is dropped once compilation finishes. Scripts now behave consistently with modules after the change introduced in #4855. **The AST is only required during parsing and compilation. Once the script finishes execution and the result is consumed, it can be safely released to reduce memory usage in long-running runtimes.** --------- Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com> Co-authored-by: José Julián Espina <julian.espina@canonical.com> Co-authored-by: José Julián Espina <jedel0124@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a memory retention issue in the module system where the parsed AST (
boa_ast::Module) and the module source text were kept in memory even after compilation finished.These structures are only required during the module initialization phase (
initialize_environment). However, they were previously stored as permanent fields insideModuleCode, meaning every loaded module kept its entire AST and source text allocated for the lifetime of the process.In workloads that load many modules, this can lead to steady memory growth over time.
Where the Issue Occurred
The problem originates in:
ModuleCodepreviously stored the full AST and source text directly:Both values are only used inside
SourceTextModule::initialize_environment(), but they remained allocated afterward.Since modules are cached by
SimpleModuleLoader, the AST and source text would stay in memory indefinitely.Fix
The AST and source text are now stored as optional values so they can be consumed and dropped once compilation is complete.
They are extracted during initialization:
After this point, the AST and source text are no longer retained by the module.
Impact
This prevents modules from permanently holding large AST structures that are no longer needed after compilation.
In module-heavy workloads (such as large dependency graphs or repeated module loading), this keeps memory usage stable instead of growing with every loaded module.
Result