Release buffered libaviutil packets

[JanuszL]

• certain libaviutils functions that read the video packets
  buffers it for further use. As DALI keeps libaviutils context
  for each video file it processes it may lead to unnecessarily
  high memory consumption. This PR releases the buffered packets
  when the iteration with the file ends.
• relates to memory use continuously increasing #4675

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

• certain libaviutils functions that read the video packets
  buffers it for further use. As DALI keeps libaviutils context
  for each video file it processes it may lead to unnecessarily
  high memory consumption. This PR releases the buffered packets
  when the iteration with the file ends.

Additional information:

Affected modules and functionalities:

• video reader

Key points relevant for the review:

• NA

Tests:

• Existing tests apply
  • test_video_pipeline.py
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[JanuszL]

!build

[dali-automaton]

CI MESSAGE: [7462821]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [7462821]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [7462821]: BUILD PASSED

[zsz00]

Can you put this pr in v1.24.0 or v1.23.1, and have a realse ?

[JanuszL]

Hi @zsz00,

We don't plan to release 1.23.1, but it should make into 1.24. In the mean time, you can try out the nightly build created after the change has been merged.

[zsz00]

xxx@VM-1-8-ubuntu:~$ pip install --extra-index-url https://developer.download.nvidia.com/compute/redist/nightly --upgrade nvidia-dali-nightly-cuda110
Looking in indexes: https://mirrors.aliyun.com/pypi/simple, https://pypi.ngc.nvidia.com, https://developer.download.nvidia.com/compute/redist/nightly
Collecting nvidia-dali-nightly-cuda110
Downloading https://developer.download.nvidia.cn/compute/redist/nightly/nvidia-dali-nightly-cuda110/nvidia_dali_nightly_cuda110-1.24.0.dev20230223-7399572-py3-none-manylinux2014_x86_64.whl (485.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━ 436.9/485.4 MB 1.3 MB/s eta 0:00:39

nvidia_dali_nightly_cuda110-1.24.0.dev20230223-7399572 Is this the latest nightly build ??

[JanuszL]

Hi @zsz00,

Yes, this is the latest available for now. Due to other issues, the CI test didn't pass so the nightly hasn't been published since then. We hope to have working nightly available soon.

[zsz00]

Ok, I will wait for it.

[zsz00]

Is nightly available now ?

[JanuszL]

I see one from 3/9 for cuda12, buf for CUDA11 the last successful one is from 2/22.