How to bound the storage automatically to the maximum required space to avoid dynamic memory allocations?

[mcourteaux]

Right now, after optimizing my pipeline, malloc still takes about 15% of the time, for the blocks of data along the border.
I'm totally fine with the memory being allocated for this be constant to fit the requirements of the normal block size (as the border blocks are usually smaller (can they ever not be?)).
It seems that whatever LLVM does if the size of the allocate is constant, no malloc() is involved, which speeds up the pipeline a fair bit.

I know you can bound_storage() to manually set this, but that requires me to calculate what size is required, whereas Halide clearly knows it, because if I put it too small, it complains that an assertion will fail at runtime. So, is there a way to put the allocation sizes constant and let Halide figure out what that minimum required size is?

Probably @abadams know this well.

[vksnk]

I know you can bound_storage() to manually set this, but that requires me to calculate what size is required, whereas Halide clearly knows it, because if I put it too small, it complains that an assertion will fail at runtime.

This is only true in some cases, because allocation can depend on a parameter which value is only known at runtime.

If you're compiling for one of the platforms which goes through CodeGen_LLVM, I think the solution is to use .store_in(MemoryType::Stack) which would use a pseudostack implementation for large allocations.

[mcourteaux]

I tried this. I put all the functions that were dynamic in size to stack, and the amount of time spent in malloc was the same. I'm wondering if the pseudostack thing you mention is one layer of abstraction to malloc. I saw a lot of mallocs in the assembly with assembly block that had comments regarding "pseudostack".

[mcourteaux]

Dumping my thoughts here: I am not sure if the Halide built-in profiler actually counts the number of mallocs correctly. Will investigate tomorrow.

[vksnk]

An alternative strategy might be to use hoist_storage() (once #7915 is submitted) to hoist allocations from the inner loop.