what is pipeline_threshold used for?

[Jack47]

I want to figure out what does pipeline and pipeline_threshold means in actnn. Din't find examples in test or readme, so may you guys give some examples or explain it a bit? Thanks

PS: I'm currently reading actnn source code and learned a lot from it , my chinese notes here.

[merrymercy]

For large tensors, we compute the forward and quantize activations one micro-batch by one micro-batch, so we call this "pipeline". This is used to reduce the temporary workspace memory and reduce memory fragmentation.

If the tensor size is larger than pipeline_threshold, we will apply this optimization.

[Jack47]

Sounds interesting, thanks for clear explain, it helps a lot.