[minor] Protect against broken/missing torchvision installations and do not hard-fail at timm/torchvision import (many text models don't need any timm/torchvision as hard dependencies)

[vadimkantorov]

Also, transformers import fails when no torchvision is installed at all. I think it should be no-error, especially if I'm working with text models only. timm/friends should not be imported at all...

System Info

Google Colab, uses pytorch 2.6.0 for now

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Go to colab which currently still has pytorch 2.6.0, then uninstall pytorch and install new pytorch (without torchvision):

!pip uninstall torch -y
!pip install torch --index-url https://download.pytorch.org/whl/cpu

`from transformers import AutoModel; AutoModel.from_pretrained('Qwen/Qwen2.5-0.5B', trust_remote_code=True)`

Currently (nothing to do with vision) fails when the setup has broken installation of torchvision (can happen if the torchvision isn't actually used and has a version not matching torch)

import transformers.models.timm_wrapper.configuration_timm_wrapper because of the following error (look up to see its traceback):
operator torchvision::nms does not exist

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py](https://localhost:8080/#) in _get_module(self, module_name)
   1966         try:
-> 1967             return importlib.import_module("." + module_name, self.__name__)
   1968         except Exception as e:

24 frames
[/usr/lib/python3.11/importlib/__init__.py](https://localhost:8080/#) in import_module(name, package)
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 

/usr/lib/python3.11/importlib/_bootstrap.py in _gcd_import(name, package, level)

/usr/lib/python3.11/importlib/_bootstrap.py in _find_and_load(name, import_)

/usr/lib/python3.11/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

/usr/lib/python3.11/importlib/_bootstrap.py in _load_unlocked(spec)

/usr/lib/python3.11/importlib/_bootstrap_external.py in exec_module(self, module)

/usr/lib/python3.11/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

[/usr/local/lib/python3.11/dist-packages/transformers/models/timm_wrapper/configuration_timm_wrapper.py](https://localhost:8080/#) in <module>
     24 if is_timm_available():
---> 25     from timm.data import ImageNetInfo, infer_imagenet_subset
     26 

[/usr/local/lib/python3.11/dist-packages/timm/__init__.py](https://localhost:8080/#) in <module>
      1 from .version import __version__ as __version__
----> 2 from .layers import (
      3     is_scriptable as is_scriptable,

[/usr/local/lib/python3.11/dist-packages/timm/layers/__init__.py](https://localhost:8080/#) in <module>
      7 from .blur_pool import BlurPool2d, create_aa
----> 8 from .classifier import create_classifier, ClassifierHead, NormMlpClassifierHead, ClNormMlpClassifierHead
      9 from .cond_conv2d import CondConv2d, get_condconv_initializer

[/usr/local/lib/python3.11/dist-packages/timm/layers/classifier.py](https://localhost:8080/#) in <module>
     14 from .create_act import get_act_layer
---> 15 from .create_norm import get_norm_layer
     16 

[/usr/local/lib/python3.11/dist-packages/timm/layers/create_norm.py](https://localhost:8080/#) in <module>
     13 from .norm import GroupNorm, GroupNorm1, LayerNorm, LayerNorm2d, RmsNorm, RmsNorm2d, SimpleNorm, SimpleNorm2d
---> 14 from torchvision.ops.misc import FrozenBatchNorm2d
     15 

[/usr/local/lib/python3.11/dist-packages/torchvision/__init__.py](https://localhost:8080/#) in <module>
      9 from .extension import _HAS_OPS  # usort:skip
---> 10 from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils  # usort:skip
     11 

[/usr/local/lib/python3.11/dist-packages/torchvision/_meta_registrations.py](https://localhost:8080/#) in <module>
    162 
--> 163 @torch.library.register_fake("torchvision::nms")
    164 def meta_nms(dets, scores, iou_threshold):

[/usr/local/lib/python3.11/dist-packages/torch/library.py](https://localhost:8080/#) in register(func)
   1022             use_lib = lib
-> 1023         use_lib._register_fake(op_name, func, _stacklevel=stacklevel + 1)
   1024         return func

[/usr/local/lib/python3.11/dist-packages/torch/library.py](https://localhost:8080/#) in _register_fake(self, op_name, fn, _stacklevel)
    213 
--> 214         handle = entry.fake_impl.register(func_to_register, source)
    215         self._registration_handles.append(handle)

[/usr/local/lib/python3.11/dist-packages/torch/_library/fake_impl.py](https://localhost:8080/#) in register(self, func, source)
     30             )
---> 31         if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
     32             raise RuntimeError(

RuntimeError: operator torchvision::nms does not exist

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
[<ipython-input-6-8843fc8bd8f0>](https://localhost:8080/#) in <cell line: 0>()
      4 with torch.device('meta'):
      5     #from torch.nn.attention.flex_attention import BlockMask, flex_attention
----> 6     AutoModel.from_pretrained('Qwen/Qwen2.5-0.5B', trust_remote_code=True)

[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    546 
    547         has_remote_code = hasattr(config, "auto_map") and cls.__name__ in config.auto_map
--> 548         has_local_code = type(config) in cls._model_mapping.keys()
    549         trust_remote_code = resolve_trust_remote_code(
    550             trust_remote_code, pretrained_model_name_or_path, has_local_code, has_remote_code

[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in keys(self)
    785 
    786     def keys(self):
--> 787         mapping_keys = [
    788             self._load_attr_from_module(key, name)
    789             for key, name in self._config_mapping.items()

[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in <listcomp>(.0)
    786     def keys(self):
    787         mapping_keys = [
--> 788             self._load_attr_from_module(key, name)
    789             for key, name in self._config_mapping.items()
    790             if key in self._model_mapping.keys()

[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in _load_attr_from_module(self, model_type, attr)
    782         if module_name not in self._modules:
    783             self._modules[module_name] = importlib.import_module(f".{module_name}", "transformers.models")
--> 784         return getattribute_from_module(self._modules[module_name], attr)
    785 
    786     def keys(self):

[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in getattribute_from_module(module, attr)
    698     if isinstance(attr, tuple):
    699         return tuple(getattribute_from_module(module, a) for a in attr)
--> 700     if hasattr(module, attr):
    701         return getattr(module, attr)
    702     # Some of the mappings have entries model_type -> object of another model type. In that case we try to grab the

[/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py](https://localhost:8080/#) in __getattr__(self, name)
   1953             value = Placeholder
   1954         elif name in self._class_to_module.keys():
-> 1955             module = self._get_module(self._class_to_module[name])
   1956             value = getattr(module, name)
   1957         elif name in self._modules:

[/usr/local/lib/python3.11/dist-packages/transformers/utils/import_utils.py](https://localhost:8080/#) in _get_module(self, module_name)
   1967             return importlib.import_module("." + module_name, self.__name__)
   1968         except Exception as e:
-> 1969             raise RuntimeError(
   1970                 f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1971                 f" traceback):\n{e}"

RuntimeError: Failed to import transformers.models.timm_wrapper.configuration_timm_wrapper because of the following error (look up to see its traceback):
operator torchvision::nms does not exist

Expected behavior

no hard-fail at import time, maybe a hard-fail when these ops/modules from torchvision are actually needed at runtime

[Rocketknight1]

cc @qubvel @rwightman when I try this I get Could not import module 'TimmWrapperConfig'. Are this object's requirements defined correctly?. It seems like maybe the Timm classes aren't being guarded / replaced with dummies correctly when torchvision is unavailable?

[vadimkantorov]

Yeah, HF currently hard-fails when either torchvision can'be imported (e.g. because of ABI issues / or incompatible PyTorch) or when torchvision isn't installed at all.

[qubvel]

Hmmm, that's probably because an outdated/incompatible version of torchvision

From the traceback:

[/usr/local/lib/python3.11/dist-packages/transformers/models/timm_wrapper/configuration_timm_wrapper.py](https://localhost:8080/#) in <module>
     24 if is_timm_available():
---> 25     from timm.data import ImageNetInfo, infer_imagenet_subset
     26 

It seems like timm is installed but can't be imported. That's one of the issues with our guards - we only check package exists without actually importing it. I would say it's an environment misconfiguration, but we can think about how to handle it.

@vadimkantorov you should not actually update only torch, torchvision should be updated accordingly, or removed

[vadimkantorov]

Hmmm, that's probably because an outdated/incompatible version of torchvision

Certainly. But unfortunately it's kind of easy to get broken installation of torchvision when upgrading torch manually :(

I would say it's an environment misconfiguration, but we can think about how to handle it.

If I remember well, it also failed when I removed torchvision completely

but we can think about how to handle it.

Yeah, I think guarding aginst torchvision/timm failing to import would be helpful - given that now so many usages are language-only...

[rwightman]

Are both timm and torchvision removed from env?

…

On Mon, May 12, 2025, 10:16 a.m. Vadim Kantorov ***@***.***> wrote: *vadimkantorov* left a comment (huggingface/transformers#38065) <#38065 (comment)> Hmmm, that's probably because an outdated/incompatible version of torchvision Certainly I would say it's an environment misconfiguration, but we can think about how to handle it. If I remember well, it also failed when I removed torchvision completely — Reply to this email directly, view it on GitHub <#38065 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLQICHE4VR4NXU4EFYLWAD26DJO5AVCNFSM6AAAAAB43FIOIGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZTGM2TINZTGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[vadimkantorov]

@rwightman they were not removed. But it's easy to get such a broken setup: start with an install of torch/torchvision and then manually update only torch (because not caring about vision stuff, when need only LLM packages to be working)

I imagine if one removes timm/torchvision completely, the problem would disappear

For me what was strange is that timm/torchvision is being imported by HF even if no vision model is used (probably this can also slow down a bit the import time), and that broken torchvision/timm can prevent transformers from being imported altogether

[qubvel]

Hey @vadimkantorov, it's not always easy to avoid unnecessary imports, but we would appreciate it if you could explore possible solutions to prevent timm/torchvision imports if text models are used 🤗

[vadimkantorov]

the easiest might be moving import of timm/torchvision behind a try-catch + print a warning when they cannot be imported instead of a hard fail... maybe the better thing would be somehow to make import of timm lazy, and do it only inside the modules/files actually using it...

e.g. maybe making it:

def is_timm_available():
   try:
      import timm
      return True
   except:
      # todo: maybe some filtering of error messages / print a warning
   return False

also, not sure why timm_wrapper/configuration_timm_wrapper.py got invoked on the main import transformers code path...

[vadimkantorov]

Maybe from .timm_backbone import *; from .timm_wrapper import * could be lazy-imported only for vision models somehow?