Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Experience-Change Logging to WandBLogger #875

Merged
merged 5 commits into from Jan 10, 2022

Conversation

HamedHemati
Copy link
Collaborator

This PR adds experience-change logging to the WandB logger by explicitly setting the step in wandb.log(.). This can be useful to have more meaningful visualizations in some of the plots.

Example:
Screenshot 2022-01-09 at 18 28 44

is mapped to:
Screenshot 2022-01-09 at 18 28 08
)

@HamedHemati HamedHemati changed the title Fixes wandb Add Experience-Change Logging to WandBLogger Jan 9, 2022
@coveralls
Copy link

coveralls commented Jan 9, 2022

Pull Request Test Coverage Report for Build 1676850052

  • 3 of 16 (18.75%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.05%) to 79.9%

Changes Missing Coverage Covered Lines Changed/Added Lines %
avalanche/logging/wandb_logger.py 3 16 18.75%
Totals Coverage Status
Change from base Build 1667756401: -0.05%
Covered Lines: 11560
Relevant Lines: 14468

💛 - Coveralls

@AndreaCossu
Copy link
Collaborator

Hi @HamedHemati ! I haven't tried your changes, yet. If I got this correctly, the user will be able to modify the x-axis interactively on W&B platform by either leaving it as default (minibatch ID) or by selecting experience ID, right?

@HamedHemati
Copy link
Collaborator Author

HamedHemati commented Jan 10, 2022

Hi @HamedHemati ! I haven't tried your changes, yet. If I got this correctly, the user will be able to modify the x-axis interactively on W&B platform by either leaving it as default (minibatch ID) or by selecting experience ID, right?

Hi @AndreaCossu. Yes exactly. By default, the x-axis in all plots remains as "step" (which is the mini-batch increments) but for plots based on streams (eg. acc/loss for test streams), you can change it to experience ID. I use this very often, so I thought it might be useful to be merged.

@AndreaCossu
Copy link
Collaborator

Great, thanks 😄 I think it can be useful, especially for long streams. Can you just change the name to TrainingExperience instead of Experience? In this way we avoid confusion with test experiences ID.

@HamedHemati
Copy link
Collaborator Author

Yea sure, done :)

@AndreaCossu AndreaCossu merged commit 138ee14 into ContinualAI:master Jan 10, 2022
@HamedHemati HamedHemati deleted the fixes_wandb branch May 2, 2022 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants