transformers require torch >= 2.1.0 to run fp8 model, but im using 2.7.0

[O5-7]

System Info

python = 3.9
torch = 2.7.0+cu128
transformers = 4.51.3

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

download Qwen3-1.7B-FP8
run quick start with local model

the result is :

  File "D:\anaconda3\lib\site-packages\transformers\models\auto\auto_factory.py", line 571, in from_pretrained
    return model_class.from_pretrained(
  File "D:\anaconda3\lib\site-packages\transformers\modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
  File "D:\anaconda3\lib\site-packages\transformers\modeling_utils.py", line 4228, in from_pretrained
    hf_quantizer.validate_environment(
  File "D:\anaconda3\lib\site-packages\transformers\quantizers\quantizer_finegrained_fp8.py", line 36, in validate_environment
    raise ImportError(
ImportError: Using fp8 quantization requires torch >= 2.1.0Please install the latest version of torch ( pip install --upgrade torch )

Im using torch=2.7.0 , I will try lower version lately

Expected behavior

no error

[Rocketknight1]

Hi @O5-7, torch 2.7 should work fine. This kind of thing is usually an environment error, which is quite tricky for us to diagnose! If you can make some code that reproduces the error consistently even on other platforms (e.g. Colab) with torch 2.7, please let us know.

[O5-7]

Hi @O5-7, torch 2.7 should work fine. This kind of thing is usually an environment error, which is quite tricky for us to diagnose! If you can make some code that reproduces the error consistently even on other platforms (e.g. Colab) with torch 2.7, please let us know.

Thank you. I found out that importlib caused this issue, which returns the wrong version of Torch.
In transformers\quantizers\quantizer_finegrained_fp8.py line 37 :

version.parse(importlib.metadata.version("torch")) < version.parse("2.1.0"):

test script:

import torch
import transformers
from importlib.metadata import version

print(torch.__version__)
print(transformers.__version__)

print(version("torch"))
print(version("transformers"))

and result:

2.6.0+cu124
4.51.3
1.12.1
4.51.3

I installed different versions of Torch and upgraded importlib, but importlib still returns 1.12.1. and there is no envs in my PC had installed torch=1.12.1
I'm not sure whether it's an issue with my env or with importlib or with torch.
And I dont have hardware to support follow test, becasue the compute capability of my GPU is too low for FP8

[Rocketknight1]

Hi @O5-7, I'm afraid this is definitely some kind of environment error! I'm not sure why importlib.metadata.version is returning the wrong value, but it's not something we can reproduce at our end.

I'm closing for now, but if you can figure out the cause, maybe we can update our version checks to avoid this in future!