Improve iterator checkpointing

[szkarpinski]

Category:

New feature (non-breaking change which adds functionality)
Refactoring (Redesign of existing code that doesn't affect functionality)

Description:

In this PR I explicitly save iterator data to the checkpoint and then use it to restore. Previously, I attempted to calculate iterator state based on pipeline state, which turned out to be a bad idea.

Iterator checkpointing now works as follows:

1. User calls it.checkpoints()
2. (!) The iterator creates a dict with values of its internal counters, pickles it and saves it to pipe._iterator_data in each of its pipelines.
3. Iterator calls pipe.checkpoint() for each of its pipelines (pipe._iterator_data is now saved in a checkpoint as well)
4. Later, user restores the pipelines and creates a new iterator.
5. (!) The iterator notices that the pipelines are restored, so it grabs pipe._iterator_data and restores its counters

This approach enables the following new features, which were problematic before:

• Checkpointing for FILL + pad_last_batch=False, which was not supported
• Reflecting .reset() in the checkpoint. Before, the fact that user has called it.reset() was not reflected in a checkpoint in any way, resulting in ambiguous restore.

Additional information:

Affected modules and functionalities:

ExternalContext, Base iterator, C API

Key points relevant for the review:

Tests:

• Existing tests apply
• New tests added
  • Python tests (test_reset to check if it.reset() is now checkpointed correctly)
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: DALI-3870

[szkarpinski]

Adding

daliDestroyExternalContextCheckpoint(&mock_external_context);

fixes a leak which was there before, introduced with #5356

[szkarpinski]

!build

[dali-automaton]

CI MESSAGE: [13612650]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [13612650]: BUILD PASSED