Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights - #46942
Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights#46942JOhnsonKC201 wants to merge 1 commit into
Conversation
`TimesFm2_5ResidualBlock.forward` casts its (float) input to `self.input_layer.weight.dtype`. Under bitsandbytes 4/8-bit quantization that weight dtype is an integer type (e.g. uint8), so the cast silently truncates the float inputs to zeros (0.73 -> 0), producing garbage outputs with no error or warning. This block is used as the model's input projection (`input_ff_layer`), so the raw time-series inputs are affected. Only cast when the weight dtype is floating point, matching the fix already applied to gemma4_unified (huggingface#46904) and gemma4 (huggingface#46933).
|
[For maintainers] Suggested jobs to run (before merge) run-slow: timesfm2_5 |
|
CI Dashboard: View test results in Grafana |
|
Small, self-contained fix (+10/-4, with Happy to run |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
TimesFm2_5ResidualBlock.forwardunconditionally casts its float input to the block's weight dtype:input_layeris annn.Linear, and this block is used as the model's input projection (input_ff_layer), so the raw float time-series inputs flow through it. When the model is loaded with quantization (e.g.bitsandbytes4/8-bit),input_layer.weight.dtypeis an integer type (e.g.uint8). Casting the float inputs to an int dtype silently truncates them to zeros (0.73 -> 0), so the model produces garbage outputs with no error or warning.The fix only casts when the weight dtype is floating point, preserving the original mixed-precision behavior while skipping the destructive cast under quantization:
This is the same pattern already applied to the multimodal embedders in
gemma4_unified(#46904) andgemma4(#46933).Both
modular_timesfm2_5.pyand the generatedmodeling_timesfm2_5.pyare updated to stay in sync.Before submitting
Who can review?
@kashif @SunMarc