Please replace .cuda() with to(device) in Stanza

[SGombert]

I wanted to test Stanza on the recently released MPS backend for the M1 macs. However, I noticed looking in the code that Stanza consistently uses the .cuda() method instead of the more flexible .to(device), and .cuda() does not seem to be overwritten by the MPS backend. While I get that it is convenient for most users to just set use_gpu=True, I miss the option to alternatively set a specific device which would make the framework much more flexible.

[AngledLuffa]

This is a good idea. I will see what we can do

…

On Thu, May 19, 2022 at 8:34 AM Sebastian Gombert ***@***.***> wrote: I wanted to test Stanza on the recently released MPS backend for the M1 macs. However, I noticed looking in the code that Stanza consistently uses the .cuda() method instead of the more flexible .to(device), and .cuda() does not seem to be overwritten by the MPS backend. While I get that it is convenient for most users to just set use_gpu=True, I miss the option to alternatively set a specific device which would make the framework much more flexible. — Reply to this email directly, view it on GitHub <#1032>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2AYWP6JZHTSZLOCJXI2IDVKZNPRANCNFSM5WMUI3YA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AngledLuffa]

I did in fact replace cuda() with to(device) everywhere:

#1159

You can now theoretically set device=... to anything on a Pipeline when you create one. However, the MPS backend has several bugs, including one which makes it completely unusable for our project:

pytorch/pytorch#80306
https://discuss.pytorch.org/t/lstm-output-transposed/154820/5

So technically I've done exactly what you wanted, while at the same time I've accomplished absolutely nothing

[AngledLuffa]

This is now part of Stanza 1.5. As mentioned above, the current torch version is buggy for MPS. The next version is supposed to have a fix for that, though

[venkatasg]

Once this is fixed in PyTorch, I assume device='ops' should be enough for Stanza to use MPS during inference?

[AngledLuffa]

mps but otherwise yes

i tried a couple weeks ago, and it didn't work with the nightly build of pytorch.

pytorch/pytorch#97552