Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve checkpointing docs #5142

Merged
merged 3 commits into from
Nov 6, 2023
Merged

Conversation

szkarpinski
Copy link
Collaborator

@szkarpinski szkarpinski commented Nov 3, 2023

Category:

Other - Documentation

Description:

  • Added a small example of checkpointing usage to pipeline docs
  • Fixed typos in checkpointing docs and removed out of date statements about checkpointning only at the beginning of an epoch
  • Fixed a mistake in enable_memory_stats docs

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>

p = pipeline()
p.build()
# Run the pipeline...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: At first I thought the Run the pipeline refers to the line below, where we save the checkpoint. Maybe adding p.run() or even some for loop to make the example more explicit would help disambiguate it?

Or mabe it's just me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I changed it to explicitly running p.run()

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10599547]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10599547]: BUILD FAILED

@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10606677]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10606677]: BUILD PASSED

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
@szkarpinski
Copy link
Collaborator Author

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10608374]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10608374]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [10608374]: BUILD PASSED

@szkarpinski szkarpinski merged commit b3a6893 into NVIDIA:main Nov 6, 2023
5 checks passed
stiepan pushed a commit that referenced this pull request Nov 6, 2023
Checkpointing docs were updated and extended with a small example

---------

Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants