pytorch_utils.py > isin_mps_friendly > RuntimeError: Expected elements.dtype() == test_elements.dtype() to be true, but got false.

[f2janyway]

I just followed this tutorial. but got error

https://huggingface.co/learn/audio-course/en/chapter2/tts_pipeline

from transformers import pipeline

pipe = pipeline("text-to-speech", model="suno/bark-small")

text = "Ladybugs have had important roles in culture and religion, being associated with luck, love, fertility and prophecy. "
output = pipe(text)

RuntimeError: Expected elements.dtype() == test_elements.dtype() to be true, but got false.

transformers/src/transformers/pytorch_utils.py

Line 334 in 7ecc5b8

return torch.isin(elements, test_elements)

so I just added this code. and works fine. when I took a log with bug code the elements.dtype shown 'int32' but others elements.dtype, and test_elements.dtype 'int64'

        if elements.dtype != test_elements.dtype:
            elements = elements.to(dtype=test_elements.dtype)
        # Note: don't use named arguments in `torch.isin`, see https://github.com/pytorch/pytorch/issues/126045
        return torch.isin(elements, test_elements)

my env:

macbook pro m1

conda

transformers-cli env
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• transformers version: 4.51.1
• Platform: macOS-15.3.1-arm64-arm-64bit
• Python version: 3.10.12
• Huggingface_hub version: 0.30.2
• Safetensors version: 0.5.3
• Accelerate version: 1.6.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (GPU?): 2.6.0 (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: No. In fact. I don't know. I didn't set about these.

Who can help?

@gante

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

just run above code.

Expected behavior

RuntimeError

[manueldeprada]

Hey @f2janyway , thanks for the report! We have had similar issues where MPS initialises tensor by default to int32 instead of int64. We should catch where this initialization is happening instead of casting them in isin_mps_friendly() :)

Could you please write this into isin_mps_friendly():

if elements.dtype != test_elements.dtype:
        import traceback
        traceback.print_stack()

so that we know which call is causing the dtype mismatch?

Thanks!

[f2janyway]

Ok. Thanks

[manueldeprada]

I meant if you could run it locally and let me know here :)

[f2janyway]

Aha Ok.

[f2janyway]

transformers/src/transformers/models/bark/modeling_bark.py

Line 900 in 953196a

torch.ones((batch_size, max_input_semantic_length + 1), dtype=torch.int, device=self.device),

torch.int is int32

When I changed to torch.int64. it works fine.

(and self.device is "mps:0")

---

RuntimeError Traceback (most recent call last)
Cell In[3], line 2
1 text = "Ladybugs have had important roles in culture and religion, being associated with luck, love, fertility and prophecy. "
----> 2 output = pipe(text)

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pipelines/text_to_audio.py:187, in TextToAudioPipeline.call(self, text_inputs, **forward_params)
165 def call(self, text_inputs: Union[str, List[str]], **forward_params):
166 """
167 Generates speech/audio from the inputs. See the [TextToAudioPipeline] documentation for more information.
168
(...)
185 - sampling_rate (int) -- The sampling rate of the generated audio waveform.
186 """
--> 187 return super().call(text_inputs, **forward_params)

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pipelines/base.py:1379, in Pipeline.call(self, inputs, num_workers, batch_size, *args, **kwargs)
1371 return next(
1372 iter(
1373 self.get_iterator(
(...)
1376 )
1377 )
1378 else:
-> 1379 return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pipelines/base.py:1387, in Pipeline.run_single(self, inputs, preprocess_params, forward_params, postprocess_params)
1385 model_inputs = self.preprocess(inputs, **preprocess_params)
1386 print("run_single/model_inputs:",model_inputs)
-> 1387 model_outputs = self.forward(model_inputs, **forward_params)
1388 outputs = self.postprocess(model_outputs, **postprocess_params)
1389 return outputs

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pipelines/base.py:1286, in Pipeline.forward(self, model_inputs, **forward_params)
1284 with inference_context():
1285 model_inputs = self._ensure_tensor_on_device(model_inputs, device=self.device)
-> 1286 model_outputs = self._forward(model_inputs, **forward_params)
1287 model_outputs = self._ensure_tensor_on_device(model_outputs, device=torch.device("cpu"))
1288 else:

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pipelines/text_to_audio.py:149, in TextToAudioPipeline._forward(self, model_inputs, **kwargs)
146 # generate_kwargs get priority over forward_params
147 forward_params.update(generate_kwargs)
--> 149 output = self.model.generate(**model_inputs, **forward_params)
150 else:
151 if len(generate_kwargs):

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/torch/utils/_contextlib.py:116, in context_decorator..decorate_context(*args, **kwargs)
113 @functools.wraps(func)
114 def decorate_context(*args, **kwargs):
115 with ctx_factory():
--> 116 return func(*args, **kwargs)

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/models/bark/modeling_bark.py:1748, in BarkModel.generate(self, input_ids, history_prompt, return_output_lengths, **kwargs)
1746 print("BarkMdoel/history_prompt",history_prompt)
1747 print("BarkMdoel/semantic_generation_config",semantic_generation_config)
-> 1748 semantic_output = self.semantic.generate(
1749 input_ids,
1750 history_prompt=history_prompt,
1751 semantic_generation_config=semantic_generation_config,
1752 **kwargs_semantic,
1753 )
1754 print("BarkMdoel/semantic_output",semantic_output)
1756 # 2. Generate from the coarse model

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/models/bark/modeling_bark.py:900, in BarkSemanticModel.generate(self, input_ids, semantic_generation_config, history_prompt, attention_mask, **kwargs)
897 # pass input_ids in order to stay consistent with the transformers generate method even though it is not used
898 # (except to get the input seq_len - that's why we keep the first 257 tokens)
899 print("BarkSemanticModel/sematic_output/super().generate",torch.int,self.device)
--> 900 semantic_output = super().generate(
901 torch.ones((batch_size, max_input_semantic_length + 1), dtype=torch.int, device=self.device),
902 input_embeds=input_embeds,
903 logits_processor=[suppress_tokens_logits_processor, early_stopping_logits_processor],
904 generation_config=semantic_generation_config,
905 **kwargs,
906 ) # size: 10048
908 # take the generated semantic tokens
909 semantic_output = semantic_output[:, max_input_semantic_length + 1 :]

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/torch/utils/_contextlib.py:116, in context_decorator..decorate_context(*args, **kwargs)
113 @functools.wraps(func)
114 def decorate_context(*args, **kwargs):
115 with ctx_factory():
--> 116 return func(*args, **kwargs)

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/generation/utils.py:2268, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, use_model_defaults, **kwargs)
2265 generation_config.use_cache = True
2267 if not kwargs_has_attention_mask and requires_attention_mask and accepts_attention_mask:
-> 2268 model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
2269 inputs_tensor, generation_config, model_kwargs
2270 )
2271 elif kwargs_has_attention_mask:
2272 # TODO (joao): generalize this check with other types of inputs
2273 if model_input_name == "input_ids" and len(model_kwargs["attention_mask"].shape) > 2:

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/generation/utils.py:727, in GenerationMixin._prepare_attention_mask_for_generation(self, inputs_tensor, generation_config, model_kwargs)
723 if not is_input_ids:
724 return default_attention_mask
726 is_pad_token_in_inputs = (pad_token_id is not None) and (
--> 727 isin_mps_friendly(elements=inputs_tensor, test_elements=pad_token_id).any()
728 )
729 is_pad_token_not_equal_to_eos_token_id = (eos_token_id is None) or ~(
730 isin_mps_friendly(elements=eos_token_id, test_elements=pad_token_id).any()
731 )
732 can_infer_attention_mask = is_pad_token_in_inputs * is_pad_token_not_equal_to_eos_token_id

File ~/.pyenv/versions/3.10.12/lib/python3.10/site-packages/transformers/pytorch_utils.py:339, in isin_mps_friendly(elements, test_elements)
337 traceback.print_stack()
338 # Note: don't use named arguments in torch.isin, see pytorch/pytorch#126045
--> 339 return torch.isin(elements, test_elements)

RuntimeError: Expected elements.dtype() == test_elements.dtype() to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)

[manueldeprada]

Thanks a lot for the trace and for taking the time to debug this, super helpful.

I’ve tracked down the root cause:

> torch.isin(torch.tensor([1,2,3], dtype=torch.int64), torch.tensor(1,dtype=torch.int32))
tensor([ True, False, False])
> torch.isin(torch.tensor([1,2,3], dtype=torch.int64).to("mps"), torch.tensor(1,dtype=torch.int32).to("mps"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: Expected elements.dtype() == test_elements.dtype() to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

It looks like the MPS implementation of isin() is not consistent with the CPU or CUDA versions when it comes to handling mixed dtypes. This inconsistency might affect more places than just modeling_bark.py, so it likely needs a broader fix.

Pinging @pcuenca! As the MPS expert, do you think we should add an explicit dtype check + cast inside isin_mps_friendly()? Might be the cleanest workaround, but since that function is marked for deprecation, not sure if it's worth patching there.

In the meantime, I’ve submitted a bug report to PyTorch so it can be tracked upstream: pytorch/pytorch#151443

[manueldeprada]

This will be fixed in PyTorch 2.7, as tracked in pytorch/pytorch#151443.

What should we do in the meantime?
@pcuenca @gante — I suggest we implement a temporary workaround in isin_mps_friendly() and postpone its deprecation until PyTorch 2.7 becomes the minimum required version.

It’s not ideal to keep isin_mps_friendly() for that long, but given that native isin() isn't fully functional on MPS until this issue is resolved, I think it's the only choice.

Let me know if that sounds good and I will implement the fix and update the deprecation accordingly.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[gante]

@manueldeprada yeah, that seems like the way to go! (add more logic to isin_mps_friendly until we have torch>=2.7 in our requirements, i.e. ~2 years from now)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[gante]

(not complete afaik, cc @manueldeprada )