New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add auto mixed precision flag to Pytorch-based Classifiers #45
Labels
feature request
New feature request
Milestone
Comments
chschroeder
added a commit
that referenced
this issue
Feb 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Feb 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Feb 29, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Mar 1, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Mar 3, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Mar 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Mar 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder
added a commit
that referenced
this issue
Mar 24, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Feature description
Expected behavior:
There should be a boolean flag (amp=True/False) in TransformerModelArguments and SetFitModelArguments. Moreover, the flag should be added as well for KimCNNClassifier whose model arguments are hopefully adjusted to match the former two classes, otherwise this flag should be in the constructor as a temporary solution.
Motivation
This could provide a nice speedup for use cases where mixed precision is good enough.
Additional comments
Note: I tried this once in a hacky way and stopped because I got nan values in the loss (and I did not really need this feature at that time). This was likely a Pytorch bug and should be kept in mind here:
https://github.com/pytorch/pytorch/releases/tag/v2.0.1
The text was updated successfully, but these errors were encountered: