Investigating Possible Memory Leak with Executor

[Aidan63]

Hello,
Appologies if the issue ends up not being in your library, I'm trying to debug a memory leak in a larger program and have managed to reproduce it in a small sample, just thought I'd run it past you first.

import haxe.io.Bytes;
import hx.concurrent.executor.Executor;

function main()
{
	final executor = Executor.create(4);
	final futures  = genFutures(executor);

	while (true)
	{
		switch Sys.stdin().readLine()
		{
			case 'exit':
				return;
			case 'free':
				for (future in futures)
				{
					future.cancel();
				}

				futures.resize(0);

				trace(cpp.vm.Gc.memUsage());

				cpp.vm.Gc.run(true);

				trace(cpp.vm.Gc.memUsage());
			case _:
				//
		}
	}
}

function genFutures(_executor : Executor)
{
	return [
		for (_ in 0...10)
		{
			final bytes = Bytes.alloc(4096 * 4096 * 4);

			_executor.submit(() -> {
				trace('captured a bytes object with ${ bytes.length } bytes');
			});
		}
	];
}

With the above program the 10 bytes objects are never collected by the GC, I've had a quick look through the threaded executor and thread pool class and it doesn't seem like they hold references to the futures after they've been ran so I'm not sure as to why the bytes objects are not collected.

My understanding is that the closure passed to the submit function will capture the bytes object, and since nothing else has a reference to it (and they won't be on the stack after the function returns) they should be eligiable for collection after the future is completed.

While the debug info and forced GC running is hxcpp specific I get the same behaviour on hashlink. I was kind of hoping writing this all out would help me understand the issue a bit more but I'm still not sure, I could just be mis-understanding how GCs work with closures. In any case it would be good to get a confirmation that the executor shouldn't be holding a reference to anything in this case.

Cheers!

[sebthom]

@Aidan63 thanks for reporting, I can confirm the leak. working on a fix...

[sebthom]

@Aidan63 I just published version 3.0.2 to lib.haxe.org which should fix the memory leak. thanks again for reporting the issue!

[Aidan63]

Yep, just gave it a go and its fixed now. Thanks!