Skip to content

Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights - #46942

Open
JOhnsonKC201 wants to merge 1 commit into
huggingface:mainfrom
JOhnsonKC201:fix-timesfm2_5-quantized-dtype-cast
Open

Fix dtype cast in TimesFM 2.5 ResidualBlock for quantized weights#46942
JOhnsonKC201 wants to merge 1 commit into
huggingface:mainfrom
JOhnsonKC201:fix-timesfm2_5-quantized-dtype-cast

Conversation

@JOhnsonKC201

Copy link
Copy Markdown

What does this PR do?

TimesFm2_5ResidualBlock.forward unconditionally casts its float input to the block's weight dtype:

x = x.to(self.input_layer.weight.dtype)

input_layer is an nn.Linear, and this block is used as the model's input projection (input_ff_layer), so the raw float time-series inputs flow through it. When the model is loaded with quantization (e.g. bitsandbytes 4/8-bit), input_layer.weight.dtype is an integer type (e.g. uint8). Casting the float inputs to an int dtype silently truncates them to zeros (0.73 -> 0), so the model produces garbage outputs with no error or warning.

The fix only casts when the weight dtype is floating point, preserving the original mixed-precision behavior while skipping the destructive cast under quantization:

if (target_dtype := self.input_layer.weight.dtype).is_floating_point:
    x = x.to(target_dtype)

This is the same pattern already applied to the multimodal embedders in gemma4_unified (#46904) and gemma4 (#46933).

Both modular_timesfm2_5.py and the generated modeling_timesfm2_5.py are updated to stay in sync.

Before submitting

Who can review?

@kashif @SunMarc

`TimesFm2_5ResidualBlock.forward` casts its (float) input to
`self.input_layer.weight.dtype`. Under bitsandbytes 4/8-bit quantization
that weight dtype is an integer type (e.g. uint8), so the cast silently
truncates the float inputs to zeros (0.73 -> 0), producing garbage
outputs with no error or warning. This block is used as the model's input
projection (`input_ff_layer`), so the raw time-series inputs are affected.

Only cast when the weight dtype is floating point, matching the fix
already applied to gemma4_unified (huggingface#46904) and gemma4 (huggingface#46933).
@github-actions

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: timesfm2_5

@github-actions

Copy link
Copy Markdown
Contributor

CI Dashboard: View test results in Grafana

@JOhnsonKC201

Copy link
Copy Markdown
Author

Small, self-contained fix (+10/-4, with modular_timesfm2_5.py and the generated modeling_timesfm2_5.py kept in sync). The destructive cast only triggers under quantization, where input_layer.weight.dtype is an integer type — so the float time-series inputs get silently truncated to zeros with no error in normal fp runs. Same failure mode and fix as the multimodal embedders in #46904 and #46933.

Happy to run run-slow: timesfm2_5 or adjust anything once a maintainer has a chance to look. Thanks!

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants