Expected GPU memory usage

[addisonklinke]

How much memory should I expect to see my GPU using for a Dali pipeline? I am running batches of 24 images where the average (compressed) image size is 58 KB in JPEG format. The pipeline includes the following operators

• contrast
• brightness
• hue
• saturation
• crop
• paste
• rotate
• resize

[JanuszL]

Hi,
All depends on the image resolution and how your pipeline looks like.
DALI is not executing in place and is processing data in batches. So we need an intermediate buffer between every operator. So for every operator n we need XnYnN (N-number of sample in the batch), where X and Y are sizes of the output image. Some operators like decoder can have big images at the output and each is different, while some like crop will have their output with a fixed size. Also, DALI provides multiple buffering so we need to add space for the output buffers as well. So as the final formula I would count the worst case sizes of images in the batch at each operator output + prefetch queue depth number of output buffers. Also, some operators, like resize need some scratch buffer on their own. We are aware that DALI memory consumption is far from perfect and we want to improve it, but not sacrificing the performance. That is why this will be part of more significant architecture rework.