You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)
Your question
Problem
On A100 GPUs, the newly added native INT8 ConvRot models (introduced in v0.27.0) are much slower than FP8 (only 1/3 to 1/2 the speed of FP8), contrary to the claimed 1.5-2x+ speedup on most NVIDIA GPUs.
Tested models:
Flux.2-Klein
Qwen-Image-Edit (qwenedit)
Environment:
GPU: NVIDIA A100 (80GB)
PyTorch: 2.10 + cu130
ComfyUI: v0.27.0)
CUDA: 13.0
Expected Behavior
According to the announcement and community benchmarks on RTX 20/30/40/50 series, INT8 ConvRot should provide:
Better or comparable quality to FP8
Significantly faster inference (especially on cards with strong INT8 Tensor Core support)
A100 has excellent INT8 Tensor Core performance, so it should benefit as well or even more.
Actual Behavior
INT8 ConvRot generation speed is only 33%~50% of FP8.
This matches reports from other users that A100 sees little to no speedup (or regression) with the current INT8 implementation.
Additional Context
ConvRot is a QuaRot variant using Hadamard rotation + row-wise INT8 to suppress outliers.
The issue is likely due to kernel fusion, dispatch paths, or missing optimizations for Ampere datacenter GPUs (vs consumer RTX cards) in the current native implementation.
Steps to Reproduce
Load an official INT8 ConvRot model (e.g. from Comfy-Org org on HF or modelscope).
Compare with the FP8 version of the same model using the same workflow.
Measure steps/second or total generation time.
Suggested Fix / Help Wanted
Improve kernel support / Triton kernels for A100/Ampere in native INT8 path.
Better fallback or mixed-precision handling for datacenter GPUs.
Profiling data or torch.compile integration improvements for ConvRot operations.
Would be great if the team could add A100-specific benchmarks or optimizations in future releases. Happy to provide more logs, profiler outputs, or test workflows.
Custom Node Testing
Your question
Problem
On A100 GPUs, the newly added native INT8 ConvRot models (introduced in v0.27.0) are much slower than FP8 (only 1/3 to 1/2 the speed of FP8), contrary to the claimed 1.5-2x+ speedup on most NVIDIA GPUs.
Tested models:
Environment:
Expected Behavior
According to the announcement and community benchmarks on RTX 20/30/40/50 series, INT8 ConvRot should provide:
A100 has excellent INT8 Tensor Core performance, so it should benefit as well or even more.
Actual Behavior
Additional Context
Steps to Reproduce
Suggested Fix / Help Wanted
Would be great if the team could add A100-specific benchmarks or optimizations in future releases. Happy to provide more logs, profiler outputs, or test workflows.
Logs
Other
No response