float: use CK stochastic rounding cuda kernel (CORE-200)#13971
float: use CK stochastic rounding cuda kernel (CORE-200)#13971rattus128 wants to merge 1 commit into
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
rattus128
requested review from
Kosinkadink,
alexisrolland,
comfyanonymous,
guill and
kijai
as code owners
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe PR adds runtime detection for comfy_kitchen.stochastic_rounding_fp8, supplying a fallback that raises NotImplementedError if unavailable. The FP8 branch in stochastic_rounding now, when the feature is present, creates an FP8-range RNG tensor from the existing torch.Generator and delegates rounding to comfy_kitchen; otherwise it uses the existing local implementation. The comfy-kitchen requirement is bumped to >=0.2.9. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@comfy/float.py`:
- Line 10: The except clause in comfy.float currently binds the exception to the
unused variable `e` (except (AttributeError, ImportError) as e:), which triggers
a lint error; update that except clause to stop binding the exception (use
except (AttributeError, ImportError):) so the exception is still caught but no
unused variable is created—locate the try/except block in comfy.float and remove
the "as e" from the except line.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 297e29f5-7a17-444f-a99d-e390796890d5
📒 Files selected for processing (2)
comfy/float.pyrequirements.txt
rattus128
force-pushed
the
prs/stochastic-kernel
branch
from
2811e01 to
0912fd5
Compare
2 participants
--- PR requires release of comfy-kitchen 0.2.9 with feature ---
Local master users can test with this wheel:
https://github.com/rattus128/comfy-kitchen/actions/runs/26069217419
Use a comfy-kitchen cuda kernel for the stochastic rounding process
Comfy-Org/comfy-kitchen#40
Dynamic-VRAM does this on the fly for offloaded loras to implement fp8mm consistently whether the weight is loaded or not (and to also unify load and offload paths). It is however performance prohibitive in the offload case.
This will also improve performance a little for primary load latency (first iteration even when fully loading).
The comfy-kitchen variant does use alternate RNG to the eager implementation it replaces so seeds do move with this change.
Example test conditions:
Windows, RTX5060, 64GB
LTX2.3 T2V 720x540x5s
Second workflow run
Before
After:
Regression Tests:
Linux, RTX5090, qwenImage 1328x1328 with 8 step lora (known very sensitive to stochastic rounding)
Before:
After:
Linux RTX5090, LTX2.3 T2V template ✅
Linux RTX5090, Ace-step 1.5 ✅
Linux RTX5090, Stable-cascade -> Flux2 ✅