Skip to content

fix(module): drop AST and source text after module initialization#4855

Merged
hansl merged 3 commits intoboa-dev:mainfrom
ashnaaseth2325-oss:fix/module-source-memory-leak
Mar 4, 2026
Merged

fix(module): drop AST and source text after module initialization#4855
hansl merged 3 commits intoboa-dev:mainfrom
ashnaaseth2325-oss:fix/module-source-memory-leak

Conversation

@ashnaaseth2325-oss
Copy link
Copy Markdown

Summary

This PR fixes a memory retention issue in the module system where the parsed AST (boa_ast::Module) and the module source text were kept in memory even after compilation finished.

These structures are only required during the module initialization phase (initialize_environment). However, they were previously stored as permanent fields inside ModuleCode, meaning every loaded module kept its entire AST and source text allocated for the lifetime of the process.

In workloads that load many modules, this can lead to steady memory growth over time.


Where the Issue Occurred

The problem originates in:

core/engine/src/module/source.rs

ModuleCode previously stored the full AST and source text directly:

struct ModuleCode {
    has_tla: bool,
    requested_modules: IndexSet<ModuleRequest>,
    source: boa_ast::Module,
    source_text: SourceText,
}

Both values are only used inside SourceTextModule::initialize_environment(), but they remained allocated afterward.

Since modules are cached by SimpleModuleLoader, the AST and source text would stay in memory indefinitely.


Fix

The AST and source text are now stored as optional values so they can be consumed and dropped once compilation is complete.

source: RefCell<Option<boa_ast::Module>>,
source_text: RefCell<Option<SourceText>>,

They are extracted during initialization:

let source = self.code.source.borrow_mut().take()
    .expect("module source consumed before initialize_environment");

After this point, the AST and source text are no longer retained by the module.


Impact

This prevents modules from permanently holding large AST structures that are no longer needed after compilation.

In module-heavy workloads (such as large dependency graphs or repeated module loading), this keeps memory usage stable instead of growing with every loaded module.


Result

  • AST memory is released immediately after module linking
  • Modules retain only the data needed for execution
  • Long-running Boa processes avoid unnecessary memory growth

ashnaaseth2325-oss added 2 commits March 4, 2026 16:19
Free the parsed boa_ast::Module and SourceText from ModuleCode once
initialize_environment() completes. Both fields are only needed during
compilation; wrapping them in RefCell<Option<_>> and taking them at the
start of that function lets the full parse tree be reclaimed
immediately after a module is linked, reducing persistent memory use in
workloads that load many modules.

Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com>
Comment thread core/engine/src/module/source.rs Outdated
let source = self
.code
.source
.borrow_mut()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to just call take() on the RefCell here. It is implemented when default is implemented which is the case for an Option.

@ashnaaseth2325-oss
Copy link
Copy Markdown
Author

Hello @hansl @jedel1043
Updated the implementation to use RefCell::take() instead of borrow_mut().take(). Thanks for pointing that out.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 4, 2026

Test262 conformance changes

Test result main count PR count difference
Total 52,862 52,862 0
Passed 49,627 49,627 0
Ignored 2,263 2,263 0
Failed 972 972 0
Panics 0 0 0
Conformance 93.88% 93.88% 0.00%

Tested PR commit: de97da8d9a3f970a008b0ba866e5607d9c744a2e

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.19%. Comparing base (6ddc2b4) to head (de97da8).
⚠️ Report is 756 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4855      +/-   ##
==========================================
+ Coverage   47.24%   57.19%   +9.94%     
==========================================
  Files         476      554      +78     
  Lines       46892    60473   +13581     
==========================================
+ Hits        22154    34585   +12431     
- Misses      24738    25888    +1150     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hansl hansl added this pull request to the merge queue Mar 4, 2026
Merged via the queue into boa-dev:main with commit 5e90957 Mar 4, 2026
19 checks passed
@jedel1043
Copy link
Copy Markdown
Member

This kinda makes it impossible to re-evaluate a module again. Might be fine I guess

ashddev pushed a commit to ashddev/boa that referenced this pull request Mar 4, 2026
…a-dev#4855)

# Summary

This PR fixes a memory retention issue in the module system where the
**parsed AST (`boa_ast::Module`) and the module source text were kept in
memory even after compilation finished**.

These structures are only required during the **module initialization
phase** (`initialize_environment`). However, they were previously stored
as permanent fields inside `ModuleCode`, meaning every loaded module
kept its entire AST and source text allocated for the lifetime of the
process.

In workloads that load many modules, this can lead to **steady memory
growth over time**.

---

# Where the Issue Occurred

The problem originates in:

```
core/engine/src/module/source.rs
```

`ModuleCode` previously stored the full AST and source text directly:

```rust
struct ModuleCode {
    has_tla: bool,
    requested_modules: IndexSet<ModuleRequest>,
    source: boa_ast::Module,
    source_text: SourceText,
}
```

Both values are only used inside
`SourceTextModule::initialize_environment()`, but they remained
allocated afterward.

Since modules are cached by `SimpleModuleLoader`, the AST and source
text would stay in memory indefinitely.

---

# Fix

The AST and source text are now stored as optional values so they can be
**consumed and dropped once compilation is complete**.

```rust
source: RefCell<Option<boa_ast::Module>>,
source_text: RefCell<Option<SourceText>>,
```

They are extracted during initialization:

```rust
let source = self.code.source.borrow_mut().take()
    .expect("module source consumed before initialize_environment");
```

After this point, the AST and source text are no longer retained by the
module.

---

# Impact

This prevents modules from permanently holding large AST structures that
are no longer needed after compilation.

In module-heavy workloads (such as large dependency graphs or repeated
module loading), this **keeps memory usage stable instead of growing
with every loaded module**.

---

# Result

* AST memory is released immediately after module linking
* Modules retain only the data needed for execution
* Long-running Boa processes avoid unnecessary memory growth

---------

Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com>
@jedel1043 jedel1043 added A-Performance Performance related changes and issues C-VM Issues and PRs related to the Boa Virtual Machine. labels Mar 6, 2026
github-merge-queue Bot pushed a commit that referenced this pull request Mar 6, 2026
## Context

This PR extends the idea introduced in #4855.

PR #4855 fixed a memory retention issue for **modules**, where the
parsed AST and source text were kept in memory even after module
initialization.

This PR applies the same idea to **scripts**, ensuring that the parsed
AST is released once script execution has completed.

# Summary

This PR prevents scripts from keeping their parsed AST after
compilation.

Right now `Script::Inner` stores the full AST in the `source` field:

```rust
struct Inner {
    realm: Realm,
    source: boa_ast::Script,
    source_text: SourceText,
    codeblock: GcRefCell<Option<Gc<CodeBlock>>>,
}
```

The AST is only needed while generating bytecode in
`Script::codeblock()`.
After compilation it is not used anymore, but it still stays in
`Script::Inner`.

Function objects created from the script keep a reference to the script
(`OrdinaryFunction::script_or_module`). Because of this, the script can
stay alive for a long time, which also keeps the whole AST in memory.


# Steps to Reproduce

In a long-running program (for example a REPL or server), repeatedly run
scripts that define functions:

```rust
let script = Script::parse(Source::from_bytes(src), None, &mut context)?;
script.evaluate(&mut context)?;
```

If those functions are stored on the global object, they keep the script
alive.
Since the script holds the AST, memory usage keeps growing as more
scripts run.


# Impact

This mostly affects long-running programs that embed Boa.

Every script that defines functions can keep its AST in memory forever.
AST structures are usually much bigger than the source code, so memory
usage can grow over time.

Modules already drop their AST after linking, but scripts did not.


# Fix

`Script::Inner::source` is changed to:

```rust
source: RefCell<Option<boa_ast::Script>>
```

The AST is used during compilation and then removed once the `CodeBlock`
is created:

```rust
*self.inner.source.borrow_mut() = None;
```


# Result

After this change, the AST is dropped once compilation finishes.

Scripts now behave consistently with modules after the change introduced
in #4855.

**The AST is only required during parsing and compilation. Once the
script finishes execution and the result is consumed, it can be safely
released to reduce memory usage in long-running runtimes.**

---------

Signed-off-by: ashnaaseth2325-oss <ashnaaseth2325@gmail.com>
Co-authored-by: José Julián Espina <julian.espina@canonical.com>
Co-authored-by: José Julián Espina <jedel0124@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Performance Performance related changes and issues C-VM Issues and PRs related to the Boa Virtual Machine.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants