-
Notifications
You must be signed in to change notification settings - Fork 618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WB-6947] Flush log data without finish with service #3137
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3137 +/- ##
==========================================
+ Coverage 79.90% 79.96% +0.06%
==========================================
Files 213 212 -1
Lines 27882 27908 +26
==========================================
+ Hits 22278 22316 +38
+ Misses 5604 5592 -12
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PERFECT!
@@ -96,9 +92,6 @@ def add_log_hooks_to_pytorch_module( | |||
if name is not None: | |||
prefix = prefix + name | |||
|
|||
if jupyter_run: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Im just curious. did you look into why this was here and what it was accomplishing before?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I couldn't find what is was doing... I saw it is used to recover history...
I guess I can keep it and just use it as a run... (if provided)
I did a simple example in a Notebook without it and it worked, but it's not a guarantee
model, | ||
log_parameters=log_parameters, | ||
log_gradients=log_gradients, | ||
prefix=prefix, | ||
log_freq=log_freq, | ||
jupyter_run=wandb.run if in_jupyter else None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is using the same wandb.run
so that's why i though it's ok to remove
Fixes WB-6947
Description
What does the PR do?
This PR moves the partial history logic to the internal process. This PR allows for runs that use the
wandb-service
path to flush their remaining history without explicitly callingwandb.finish
.This is the first PR in series of PR that move the step tracking logic from the user process to the internal process.
Testing
How was this PR tested?
This PR should keep functionality, it is tested using the existing tests.
Checklist