Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: update wandb callback to upload checkpoints #21035

Merged
merged 12 commits into from
Jan 10, 2023
Merged

feature: update wandb callback to upload checkpoints #21035

merged 12 commits into from
Jan 10, 2023

Conversation

parambharat
Copy link
Contributor

What does this PR do?

The PR updates the WandbCallback with the following changes:

  • Adds on_save method to upload model checkpoints as artifacts.
  • Changes the default value of environment variable WANDB_WATCH from gradients to false. This enables quicker training when defaults are used. The user can easily change this behavior by setting the env variable.
  • Changes the WANDB_LOG_MODEL variable from bool to str allowing for different settings to upload artifacts.
  • Modifies the class dostring to reflect the above changes.
  • Fixes broken link to wandb documentation
  • Changes the wandb run_name from output_dir to wandb auto generated name. this avoids duplication of run names in wandb workspace

Before submitting

Who can review?

Examples

  • Example colab reflecting all the changes to the WandbCallback
  • Example Weights & Biases workspace with runs that show different settings.
  • Example Weights & Biases Artifact created for checkpoints

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jan 6, 2023 •
edited

The documentation is not available anymore as the PR was closed or merged.

sgugger
sgugger reviewed Jan 7, 2023
View reviewed changes
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR! I left a couple of comments.

src/transformers/integrations.py Outdated Show resolved Hide resolved
src/transformers/integrations.py Outdated Show resolved Hide resolved
@parambharat
Copy link
Contributor Author

Hey, @sgugger : Thanks for the quick review and suggestions. I've resolved all the issues. 🤗

@sgugger
Copy link
Collaborator

sgugger commented Jan 8, 2023

@parambharat There is a weird issue with the tests. Could you try refreshing your permissions as shown here and then pushing an empty commit?

@parambharat
Copy link
Contributor Author

Hey, @sgugger. I ran the fix that you suggested and all checks pass now.

stevhliu
stevhliu approved these changes Jan 9, 2023
View reviewed changes
Copy link
Member

@stevhliu stevhliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating! I left two suggestions on formatting :)

src/transformers/integrations.py Outdated Show resolved Hide resolved
src/transformers/integrations.py Outdated Show resolved Hide resolved
parambharat and others added 2 commits January 10, 2023 10:28
@parambharat @stevhliu
fix: format deprecation doc strings review suggestion
73e873d 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@parambharat @stevhliu
docs: Highlight WANDB_DISABLED arg in documentaion
3349351 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@parambharat
Copy link
Contributor Author

parambharat commented Jan 10, 2023 •
edited

Thank you @stevhliu. I've committed your recommendations.

@sgugger sgugger merged commit 07cde58 into huggingface:main Jan 10, 2023
@sgugger
Copy link
Collaborator

sgugger commented Jan 10, 2023

Thanks again for your contribution!

venkat-natchi pushed a commit to venkat-natchi/transformers that referenced this pull request Jan 22, 2023
@parambharat @sgugger
@stevhliu @venkat-natchi
feature: update wandb callback to upload checkpoints (huggingface#21035)
d1a247a 
* docs: add wandb metrics and model checkpointing to callback docstrings

* docs: update reference to wandb documentation

* fix: change default of `"WANDB_WATCH"` from ``"gradients"` to ``"false"`

* feature: add `on_save` method and update `"WANDB_LOG_MODEL` behaviour

* fix: use default wandb run names instead of `output_dir`

- removes duplicated run names from wandb workspace
- models can be logged with corresponding run names

* fix: edit deprecation warning based on review suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix: change indentation of docstrings

* fix: change indentation of docstrings and run fixup

* fix: empty commit for circleci permissions issue

* fix: format deprecation doc strings review suggestion

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* docs: Highlight WANDB_DISABLED arg in documentaion

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix: run fixup after updating docstrings

Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
miyu386 pushed a commit to miyu386/transformers that referenced this pull request Feb 9, 2023
@parambharat @sgugger
@stevhliu @miyu386
feature: update wandb callback to upload checkpoints (huggingface#21035)
2ffa2a2 
* docs: add wandb metrics and model checkpointing to callback docstrings

* docs: update reference to wandb documentation

* fix: change default of `"WANDB_WATCH"` from ``"gradients"` to ``"false"`

* feature: add `on_save` method and update `"WANDB_LOG_MODEL` behaviour

* fix: use default wandb run names instead of `output_dir`

- removes duplicated run names from wandb workspace
- models can be logged with corresponding run names

* fix: edit deprecation warning based on review suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix: change indentation of docstrings

* fix: change indentation of docstrings and run fixup

* fix: empty commit for circleci permissions issue

* fix: format deprecation doc strings review suggestion

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* docs: Highlight WANDB_DISABLED arg in documentaion

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix: run fixup after updating docstrings

Co-authored-by: Bharat Ramanathan <ramanathan.parameshwaran@gohuddl.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@parambharat parambharat deleted the feature/wandb-callback-update branch January 10, 2024 11:00
@qubvel qubvel mentioned this pull request Apr 24, 2024
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sgugger sgugger sgugger left review comments

@stevhliu stevhliu stevhliu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@parambharat @HuggingFaceDocBuilderDev @sgugger @stevhliu