TorchAO example broken

[AbhinavGopal]

Describe the bug

The torchao example https://huggingface.co/docs/diffusers/v0.32.2/en/quantization/torchao#torchao errors with

  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/generation/utils.py", line 116, in <module>
    from accelerate.hooks import AlignDevicesHook, add_hook_to_module
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/accelerator.py", line 36, in <module>
    from accelerate.utils.imports import is_torchao_available
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py", line 14, in <module>
    from .ao import convert_model_to_fp8_ao, filter_first_and_last_linear_layers, has_ao_layers
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/ao.py", line 28, in <module>
    from torchao.float8.float8_linear import Float8LinearConfig
  File "/home/ec2-user/torchao.py", line 1, in <module>
    from diffusers import FluxPipeline
ImportError: cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 28, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/modeling_utils.py", line 48, in <module>
    from .generation import CompileConfig, GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 920, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/pipelines/flux/pipeline_flux.py", line 20, in <module>
    from transformers import (
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1852, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/torchao.py", line 1, in <module>
    from diffusers import FluxPipeline
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 910, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 922, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import diffusers.pipelines.flux.pipeline_flux because of the following error (look up to see its traceback):
Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

Reproduction

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel, TorchAoConfig

model_id = "black-forest-labs/FLUX.1-dev"
dtype = torch.bfloat16

quantization_config = TorchAoConfig("int8wo")
transformer = FluxTransformer2DModel.from_pretrained(
    model_id,
    subfolder="transformer",
    quantization_config=quantization_config,
    torch_dtype=dtype,
)
pipe = FluxPipeline.from_pretrained(
    model_id,
    transformer=transformer,
    torch_dtype=dtype,
)
pipe.to("cuda")

# Without quantization: ~31.447 GB
# With quantization: ~20.40 GB
print(f"Pipeline memory usage: {torch.cuda.max_memory_reserved() / 1024**3:.3f} GB")

prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt, num_inference_steps=50, guidance_scale=4.5, max_sequence_length=512
).images[0]
image.save("output.png")

Logs

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/generation/utils.py", line 116, in <module>
    from accelerate.hooks import AlignDevicesHook, add_hook_to_module
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/accelerator.py", line 36, in <module>
    from accelerate.utils.imports import is_torchao_available
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py", line 14, in <module>
    from .ao import convert_model_to_fp8_ao, filter_first_and_last_linear_layers, has_ao_layers
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/ao.py", line 28, in <module>
    from torchao.float8.float8_linear import Float8LinearConfig
  File "/home/ec2-user/torchao.py", line 1, in <module>
    from diffusers import FluxPipeline
ImportError: cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 28, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/modeling_utils.py", line 48, in <module>
    from .generation import CompileConfig, GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 920, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/pipelines/flux/pipeline_flux.py", line 20, in <module>
    from transformers import (
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1852, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/torchao.py", line 1, in <module>
    from diffusers import FluxPipeline
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 910, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 922, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import diffusers.pipelines.flux.pipeline_flux because of the following error (look up to see its traceback):
Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'FluxPipeline' from 'diffusers' (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/__init__.py)

System Info

• 🤗 Diffusers version: 0.32.2
• Platform: Linux-5.10.234-225.895.amzn2.x86_64-x86_64-with-glibc2.26
• Running on Google Colab?: No
• Python version: 3.11.11
• PyTorch version (GPU?): 2.6.0+cu124 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.29.3
• Transformers version: 4.49.0
• Accelerate version: 1.5.2
• PEFT version: 0.14.0
• Bitsandbytes version: not installed
• Safetensors version: 0.5.3
• xFormers version: not installed
• Accelerator: NVIDIA L40S, 46068 MiB
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: no

Who can help?

@DN6 @sayakpaul @yiyixuxu

[AbhinavGopal]

After I install torchao, when I do a simple "import accelerate", I get this log:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/generation/utils.py", line 116, in <module>
    from accelerate.hooks import AlignDevicesHook, add_hook_to_module
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/hooks.py", line 21, in <module>
    from .state import PartialState
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/state.py", line 28, in <module>
    from .utils import (
ImportError: cannot import name 'DistributedType' from partially initialized module 'accelerate.utils' (most likely due to a circular import) (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1863, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py", line 28, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/modeling_utils.py", line 48, in <module>
    from .generation import CompileConfig, GenerationConfig, GenerationMixin
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'DistributedType' from partially initialized module 'accelerate.utils' (most likely due to a circular import) (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 920, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/pipelines/flux/pipeline_flux.py", line 20, in <module>
    from transformers import (
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1852, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1851, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1865, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'DistributedType' from partially initialized module 'accelerate.utils' (most likely due to a circular import) (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/__init__.py", line 16, in <module>
    from .accelerator import Accelerator
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/accelerator.py", line 36, in <module>
    from accelerate.utils.imports import is_torchao_available
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py", line 14, in <module>
    from .ao import convert_model_to_fp8_ao, filter_first_and_last_linear_layers, has_ao_layers
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/ao.py", line 28, in <module>
    from torchao.float8.float8_linear import Float8LinearConfig
  File "/home/ec2-user/torchao.py", line 1, in <module>
    from diffusers import FluxPipeline
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 911, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 910, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/diffusers/utils/import_utils.py", line 922, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import diffusers.pipelines.flux.pipeline_flux because of the following error (look up to see its traceback):
Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
Failed to import transformers.generation.utils because of the following error (look up to see its traceback):
cannot import name 'DistributedType' from partially initialized module 'accelerate.utils' (most likely due to a circular import) (/home/ec2-user/miniconda3/envs/comfy/lib/python3.11/site-packages/accelerate/utils/__init__.py)

Figured this might be helpful

[sayakpaul]

Cc: @a-r-r-o-w

[a-r-r-o-w]

This looks like what happens to me when a venv goes bad due to lots of conflicts. What happens if you do a clean reinstall:

python3 -m venv venv
source venv/bin/activate
pip install diffusers transformers accelerate peft torch==2.5.1 torchvision torchao==0.7.0 

Also, might want to try out uv to keep your sanity with all the weird conflicts that may arise between package versions