Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auto mixed precision flag to Pytorch-based Classifiers #45

Closed
chschroeder opened this issue Aug 24, 2023 · 0 comments
Closed

Add auto mixed precision flag to Pytorch-based Classifiers #45

chschroeder opened this issue Aug 24, 2023 · 0 comments
Labels
feature request New feature request

Comments

@chschroeder
Copy link
Contributor

Feature description

Expected behavior:
There should be a boolean flag (amp=True/False) in TransformerModelArguments and SetFitModelArguments. Moreover, the flag should be added as well for KimCNNClassifier whose model arguments are hopefully adjusted to match the former two classes, otherwise this flag should be in the constructor as a temporary solution.

Motivation

This could provide a nice speedup for use cases where mixed precision is good enough.

Additional comments

Note: I tried this once in a hacky way and stopped because I got nan values in the loss (and I did not really need this feature at that time). This was likely a Pytorch bug and should be kept in mind here:
https://github.com/pytorch/pytorch/releases/tag/v2.0.1

@chschroeder chschroeder added the feature request New feature request label Aug 24, 2023
@chschroeder chschroeder added this to the small-text-2.0.0 milestone Feb 4, 2024
chschroeder added a commit that referenced this issue Feb 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Feb 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Feb 29, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Mar 1, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Mar 3, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Mar 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Mar 17, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
chschroeder added a commit that referenced this issue Mar 24, 2024
Signed-off-by: Christopher Schröder <chschroeder@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature request
Projects
None yet
Development

No branches or pull requests

1 participant