"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" #104

Alokia · 2023-02-17T04:02:51Z

按照 https://github.com/Oneflow-Inc/diffusers/wiki/How-to-Run-OneFlow-Stable-Diffusion 中 Without Docker 的方式配置，所有操作都完成，但是执行 from diffusers import OneFlowStableDiffusionPipeline 报错

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zhaodi/diffusers/src/diffusers/__init__.py", line 22, in <module>
    from transformers import CLIPTextModel, CLIPFeatureExtractor
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1101, in __getattr__
    value = getattr(module, name)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1100, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1112, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

使用这个方法可以解决导入报错：https://github.com/Oneflow-Inc/diffusers/issues/104#issuecomment-1434151151

我想要运行该样例：https://github.com/Oneflow-Inc/diffusers/blob/oneflow-fork/tests/test_pipelines_oneflow_graph_load.py
于是按照 https://github.com/Oneflow-Inc/diffusers/issues/104#issuecomment-1434151151 在 from diffusers import ... 之前先导入 transformers 模块，但是在运行中触发了新的错误

sd init time  373.8917450904846 s.
==> function  text_to_image  try to run...
text_to_image  cuda mem before  4336.75  MB
text_to_image  host mem before  9173.0  MB
E
======================================================================
ERROR: test_sd_graph_save_and_load (__main__.OneFlowPipeLineGraphSaveLoadTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/zhaodi/work/test.py", line 175, in test_sd_graph_save_and_load
    _test_sd_graph_save_and_load(True, f0 ,f1, f2)
  File "/home/zhaodi/work/test.py", line 141, in _test_sd_graph_save_and_load
    no_g_images = text_to_image(prompt, (i, j), prefix=f"is_save_{str(is_save)}-", with_graph=False)
  File "/home/zhaodi/work/test.py", line 32, in new_fn
    out = fn(*args, **kwargs)
  File "/home/zhaodi/work/test.py", line 117, in text_to_image
    images = pipe(
  File "/home/zhaodi/oneflow/python/oneflow/autograd/autograd_mode.py", line 154, in wrapper
    return func(*args, **kwargs)
  File "/home/zhaodi/diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_oneflow.py", line 620, in __call__
    text_embeddings = self._encode_prompt(
  File "/home/zhaodi/diffusers/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_oneflow.py", line 393, in _encode_prompt
    text_embeddings = self.text_encoder(text_input_ids.to(device), attention_mask=attention_mask)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 816, in forward
    return self.text_model(
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 712, in forward
    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 227, in forward
    inputs_embeds = self.token_embedding(input_ids)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 160, in forward
    return F.embedding(
  File "/home/zhaodi/miniconda3/lib/python3.10/site-packages/torch/nn/functional.py", line 2210, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor

----------------------------------------------------------------------
Ran 1 test in 374.084s

FAILED (errors=1)

System Info

Linux
oneflow 0.9.1.dev20230216+cu117
transformers 4.26.1
diffusers 0.10.0.dev0
huggingface-hub 0.12.0

The text was updated successfully, but these errors were encountered:

Alokia · 2023-02-17T06:14:01Z

我根据报错信息中的这句话 from transformers import CLIPTextModel, CLIPFeatureExtractor，做了一些尝试

执行 import diffusers，也报同样的错误
执行 from transformers import CLIPTextModel, CLIPFeatureExtractor 没有报错误
执行 from transformers import CLIPTextModel, CLIPFeatureExtractor 后，再次执行 import diffusers 不再报错误，于是我继续尝试 from diffusers import OneFlowStableDiffusionPipeline 也不再报错。
当我关闭这个 ipython，开启新的后，import diffusers 继续报错

在导入diffusers之前先导入transformers可以解决报错的问题，但不知道原因

下面附上测试过程：

In [1]: import diffusers

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

In [1]: import oneflow

In [2]: with oneflow.mock_torch.enable():
   ...:     from transformers import CLIPTextModel, CLIPFeatureExtractor

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

In [1]: from transformers import CLIPTextModel, CLIPFeatureExtractor

In [2]: import oneflow

In [3]: with oneflow.mock_torch.enable():
   ...:     from transformers import CLIPTextModel, CLIPFeatureExtractor
   ...:

In [1]: from transformers import CLIPTextModel, CLIPFeatureExtractor

In [2]: import diffusers

In [3]: from diffusers import OneFlowStableDiffusionPipeline

jackalcooper · 2023-02-17T06:26:00Z

python3 -m pip uninstall accelerate

Alokia · 2023-02-17T06:29:40Z

solved : )

strint · 2023-02-17T06:41:15Z

可以先安装下 accelerate，复现问题；

然后改下：
diffusers/src/diffusers/utils/import_utils.py

def is_accelerate_available():
    return _accelerate_available

注释下这里，让该函数返回 false，看是否能解决问题。如果可以的话，把改动可以提交到 oneflow 的 diffusers 仓库。

@Alokia

Alokia · 2023-02-17T06:49:49Z

更改后，重新执行 python3 -m pip install -e .[oneflow]，导入diffuser依旧报错

In [1]: import diffusers
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1110, in _LazyModule._get_module(self, module_name)
   1109 try:
-> 1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:

File ~/miniconda3/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:27
     26 from ...modeling_outputs import BaseModelOutput, BaseModelOutputWithPooling
---> 27 from ...modeling_utils import PreTrainedModel
     28 from ...utils import (
     29     ModelOutput,
     30     add_start_docstrings,
   (...)
     33     replace_return_docstrings,
     34 )

File ~/miniconda3/lib/python3.10/site-packages/transformers/modeling_utils.py:83
     82 if is_accelerate_available():
---> 83     from accelerate import __version__ as accelerate_version
     84     from accelerate import dispatch_model, infer_auto_device_map, init_empty_weights

File ~/miniconda3/lib/python3.10/site-packages/accelerate/__init__.py:7
      5 __version__ = "0.16.0"
----> 7 from .accelerator import Accelerator
      8 from .big_modeling import (
      9     cpu_offload,
     10     disk_offload,
   (...)
     14     load_checkpoint_and_dispatch,
     15 )

File ~/miniconda3/lib/python3.10/site-packages/accelerate/accelerator.py:29
     27 import torch.utils.hooks as hooks
---> 29 from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
     30 from .data_loader import DataLoaderDispatcher, prepare_data_loader, skip_first_batches

File ~/miniconda3/lib/python3.10/site-packages/accelerate/checkpointing.py:22
     21 import torch
---> 22 from torch.cuda.amp import GradScaler
     24 from .utils import (
     25     MODEL_NAME,
     26     OPTIMIZER_NAME,
   (...)
     32     save,
     33 )

File ~/oneflow/python/oneflow/mock_torch/__init__.py:44, in ModuleWrapper.__getattr__(self, name)
     43         return [attr for attr in dir(self.module) if not attr.startswith("_")]
---> 44     raise ModuleNotFoundError(self.module.__name__ + "." + name + error_msg)
     45 attr = getattr(self.module, name)

ModuleNotFoundError: oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[1], line 1
----> 1 import diffusers

File ~/diffusers/src/diffusers/__init__.py:22
     20 import oneflow
     21 with oneflow.mock_torch.enable():
---> 22     from transformers import CLIPTextModel, CLIPFeatureExtractor
     23 from .modeling_utils import ModelMixin
     24 from .models import AutoencoderKL, Transformer2DModel, UNet1DModel, UNet2DConditionModel, UNet2DModel, VQModel

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1101, in _LazyModule.__getattr__(self, name)
   1099 elif name in self._class_to_module.keys():
   1100     module = self._get_module(self._class_to_module[name])
-> 1101     value = getattr(module, name)
   1102 else:
   1103     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1100, in _LazyModule.__getattr__(self, name)
   1098     value = self._get_module(name)
   1099 elif name in self._class_to_module.keys():
-> 1100     module = self._get_module(self._class_to_module[name])
   1101     value = getattr(module, name)
   1102 else:

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1112, in _LazyModule._get_module(self, module_name)
   1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:
-> 1112     raise RuntimeError(
   1113         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1114         f" traceback):\n{e}"
   1115     ) from e

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

strint · 2023-02-17T06:58:30Z

File ~/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:27
     26 from ...modeling_outputs import BaseModelOutput, BaseModelOutputWithPooling
---> 27 from ...modeling_utils import PreTrainedModel
     28 from ...utils import (
     29     ModelOutput,
     30     add_start_docstrings,
   (...)
     33     replace_return_docstrings,
     34 )

File ~/miniconda3/lib/python3.10/site-packages/transformers/modeling_utils.py:83
     82 if is_accelerate_available():
---> 83     from accelerate import __version__ as accelerate_version
     84     from accelerate import dispatch_model, infer_auto_device_map, init_empty_weights

尴尬了，这个路径是 transformers 库里面的，我们现在让用户自己安装的官方 transformers，这样就改不了它的逻辑了。

那就还是沿用上面的方法吧：

python3 -m pip uninstall accelerate

Alokia added the Request-bug Something isn't working label Feb 17, 2023

Alokia changed the title ~~import OneFlowStableDiffusionPipeline error~~ import error Feb 17, 2023

strint changed the title ~~import error~~ import error: oneflow.cuda.amp.GradScaler is not implemented Feb 17, 2023

strint changed the title ~~import error: oneflow.cuda.amp.GradScaler is not implemented~~ "import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" Feb 17, 2023

Alokia mentioned this issue Feb 17, 2023

无法找到OneFlowStableDiffusionPipeline #103

Closed

strint closed this as completed Feb 17, 2023

strint added the sig-hfdiffusers label Apr 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" #104

"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" #104

Alokia commented Feb 17, 2023 •

edited

Alokia commented Feb 17, 2023

jackalcooper commented Feb 17, 2023 •

edited

Alokia commented Feb 17, 2023

strint commented Feb 17, 2023

Alokia commented Feb 17, 2023 •

edited

strint commented Feb 17, 2023

"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" #104

"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" #104

Comments

Alokia commented Feb 17, 2023 • edited

System Info

Alokia commented Feb 17, 2023

jackalcooper commented Feb 17, 2023 • edited

Alokia commented Feb 17, 2023

strint commented Feb 17, 2023

Alokia commented Feb 17, 2023 • edited

strint commented Feb 17, 2023

Alokia commented Feb 17, 2023 •

edited

jackalcooper commented Feb 17, 2023 •

edited

Alokia commented Feb 17, 2023 •

edited