Reduce memory use

[overlookmotel]

Hi. I read about Ganesha on OXC discussion.

I'm not familiar with how wasm-pack works, so I could be wrong, but I think the new implementation uses more memory than it needs.

If I am understanding it right, a single ModuleTransformerImpl is created when the loader initializes, and it will stay alive throughout the NodeJS process.

When ModuleTransformerImpl is created, it creates an oxc_allocator::Allocator which it holds on to until the NodeJS process exits:

ganesha/src/lib.rs

Lines 43 to 45 in a9b97d8

	pub fn new () -> ModuleTransformerImpl {
	Self(Allocator::default())
	}

oxc_allocator::Allocator is an arena allocator. It's fast because it allocates memory in large chunks, but does not free any memory until the Allocator is dropped.

So every time the loader parses a TS file, OXC parses the code to an AST. That AST is stored in the Allocator, which allocates more memory. Once transforming the file is complete, the AST has fulfilled its purpose and is not needed any more, but because you're reusing the same Allocator, the memory never gets freed.

NB: AST structures can be pretty big even for small files.

So I imagine you'd be better off creating a new Allocator each time in fn transform, and dropping it at the end of transform. Then the loader will not hold on to any memory after it's done transforming.

Hope this is helpful.

[egasimus]

Sounds about right, great catch! I'll make the necessary changes.