Bug: smith-normal-form test failure due to bytesRequested larger than one block at limit check

[shwestrick]

The smith-normal-form regression test is causing a segfault.

For example, on commit dfddbd5:

[mpl/regression]$ ../build/bin/mpl smith-normal-form.sml
[mpl/regression]$ ./smith-normal-form
...
Segmentation fault (core dumped)

I looked into it a little bit, and found that the mutator is requesting too many bytes at a limit check:

[mpl/regression]$ ../build/bin/mpl -debug true -debug-runtime true -keep g smith-normal-form.sml
[mpl/regression]$ ./smith-normal-form
...
smith-normal-form: gc/garbage-collection.c:98: GC_collect: Assertion `bytesRequested + sizeof(struct HM_chunk) <= s->controls->blockSize' failed.

In gdb, the specific bytesRequested is 4831. The default block size is 4096.

If I instead run with @mpl block-size 8K --, then the test makes more progress, but eventually still exits with a segfault, this time requesting 11983 bytes.

In principle this should be impossible because mlton/backend/limit-check.fun is supposed to ensure that less than 4K bytes are requested at limit checks:

mpl/mlton/backend/limit-check.fun

Lines 1115 to 1129 in dfddbd5

	(* SAM_NOTE: I chose 512*7 because the smallest block size that we
	* use is 4K, and each chunk has a chunk descriptor of some small
	* size at the beginning of each chunk. To be correct, the chunk
	* descriptor can be at most 512 bytes! The current runtime
	* uses chunk descriptors of size much smaller than 512 bytes.
	*
	* TODO: This is a dirty hack. Ideally, at compile time, a block
	* size should be chosen and we should pass the parameter:
	* blockSize - sizeOfChunkDescriptor
	*)
	val maxBytesAlloc =
	Bytes.- (Bytes.fromInt (512*7), Runtime.limitSlop)
	in
	insertCoalesce (f, maxBytesAlloc, handlesSignals)
	end

Should be fun to debug 😎