Questions on memory usage and configuration

[prat0088]

After spending some time reading the code, it appears default behavior is to pool everything with no upper limit. I wasn't able to find that information in the documentation.

Can you check my understanding?

MaximumFreeLargePoolBytes is never set, so the buffer gets added to the pool on dispose here.

The max potential memory usage of the large pool is

(maximumBufferSize/largeBufferMultiple) * MaximumFreeLargePoolBytes

or, stated another way:

the number of pools * MaximumFreeLargePoolBytes

If I don't call GetBuffer() and I don't call GetStream(asContiguousBuffer=true), I will only ever use small blocks.

So, if you rarely call GetBuffer(), a valid sizing strategy would be to create medium-sized small blocks (say 1/4 of the size of your expected common stream size), a large MaximumFreeSmallPoolBytes, and a MaximumFreeLargePoolBytes size of 1 byte to force unpooled large buffer allocation in the rare case you need it (if it was set to 0 large buffers would be pooled and retained indefinitely).

[benmwatson]

It's hard to set a default for such a general-purpose library--since we're talking about (usually) large amount of memory usage, the settings must be tailored to the app's circumstances.

I think your strategy would make sense--if you want to force ONLY usage of blocks, but never pool the large buffers--that's a valid strategy. It all depends on how you use the library. We've tried to make it as general-purpose as possible.

You may also want to look into the new Large-buffer allocation strategy in 1.3--it uses an exponential algorithm to decide the large pool sizes, which can lead to much smaller overall memory usage if you setup the initial values correctly. I need to work on some documentation for this, but it's easy to play around with.

[benmwatson]

Please see the updated documentation for more details. If you still have questions, let me know, and I can try to address them.