Skip to content

Conversation

@barney-s
Copy link
Contributor

@barney-s barney-s commented Nov 6, 2024

This was causing fastNLP_BERT model to fail.

https://github.com/fastnlp/fastNLP/blob/4e95989e973f59b2ecb7f718647257e8b6fea0c7/fastNLP/embeddings/bert_embedding.py#L427

467                word_pieces[i, 1:word_pieces_lengths[i] + 1] = torch.LongTensor(word_pieces_i)

torch.LongTensor was returning a torch.Tensor instead of XLA Array.

Debugging the call trace

DISPATCH: aten::lift_fresh
 FUNCTION: aten::lift_fresh

fixes: #8126

@ManfeiBai ManfeiBai merged commit 6454b42 into pytorch:master Nov 6, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model support for fastNLP_Bert with Torch_XLA2

3 participants