You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Downstream fine-tuning can run faster and use less GPU memory if fp16 is enabled in downstrean/runner.py. It is already implemented in upstream/runner.py.
It should not affect any existing code since fp16 is off by default in the downstream configs. Users would opt-in by setting runner.fp16: true in their downstream configs.
Describe the solution you'd like
Mimic the implementation of mixed precision training from upstream/runner.py in downstream/runner.py
Describe alternatives you've considered
None
Additional context
I've implemented locally so I'm happy to send a PR if you would like.
The text was updated successfully, but these errors were encountered:
Feature request
Is your feature request related to a problem? Please describe.
Downstream fine-tuning can run faster and use less GPU memory if fp16 is enabled in downstrean/runner.py. It is already implemented in upstream/runner.py.
It should not affect any existing code since fp16 is off by default in the downstream configs. Users would opt-in by setting runner.fp16: true in their downstream configs.
Describe the solution you'd like
Mimic the implementation of mixed precision training from upstream/runner.py in downstream/runner.py
Describe alternatives you've considered
None
Additional context
I've implemented locally so I'm happy to send a PR if you would like.
The text was updated successfully, but these errors were encountered: