Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TensorRT 9.1 support? #64

Closed
Andarwinux opened this issue Oct 20, 2023 · 1 comment
Closed

TensorRT 9.1 support? #64

Andarwinux opened this issue Oct 20, 2023 · 1 comment

Comments

@Andarwinux
Copy link

NVIDIA/TensorRT#3174
9.1 Added BF16 support.
WolframRhodium added a commit that referenced this issue Oct 21, 2023
@WolframRhodium
scripts/vsmlrt.py: add support for int8, bf16 and fp8 execution
1549bdf 
#64

This is an experimental feature. Users need to be careful about the loss of precision themselves. This only specifies allowed internal processing precisions and does not specify the I/O format.
@WolframRhodium
Copy link
Contributor

WolframRhodium commented Oct 21, 2023 •
edited

Verified that BF16 is available for Waifu2xModel.swin_unet_art on Ada, but not for DPIR. So the kernels might only support self-attentions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@WolframRhodium @Andarwinux