Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make RunningAverage and Rouge serializable #3035

Conversation

sadra-barikbin
Copy link
Collaborator

@sadra-barikbin sadra-barikbin commented Aug 21, 2023

Now state_dict could serialize metric attributes that themselves are metric or simple collections of metrics.(For Rouge). If an attribute is None, state_dict sets the same in the dict (for _value in RunningAverage)

Follow-up PR to #3001

@github-actions github-actions bot added the module: metrics Metrics module label Aug 21, 2023
Copy link
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sadra-barikbin, PR looks good to me!
Can you please check and report here how it works on MNIST while saving and resuming.
As a test example, please use:
https://github.com/pytorch/ignite/blob/11a1fba0594f6ac0db9e8bc5f21305dc338bb7c1/examples/mnist/mnist_save_resume_engine.py
Here are few instructions on https://github.com/pytorch/ignite/blob/master/examples/mnist/README.md#training-save--resume
Finally, we can also update mnist_save_resume_engine.py code adding RunningAverage metric for batch loss to the trainer and plotting it.

@sadra-barikbin
Copy link
Collaborator Author

Here is the running loss until stop and after resume:
image

image

@sadra-barikbin
Copy link
Collaborator Author

Regarding your comment in the main PR, I introduced a new if branch in Checkpoint to accept a dictionary of serializable objects as an item in the to_save. This way we could do

metrics = {'acc': Accuracy(), 'nll': Loss()}
to_save={'model':model, 'trainer':trainer, 'metrics': metrics}

How is this?

@vfdev-5 vfdev-5 merged commit 759dae3 into pytorch:master Oct 3, 2023
16 of 18 checks passed
@vfdev-5 vfdev-5 changed the title [Follow-up] Make RunningAverage and Rouge serializable Make RunningAverage and Rouge serializable Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: metrics Metrics module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants