Update for PyTorch v1.5 #172
Conversation
|
Cool! 👍 It seems that the functions for autocasting in auto mixed precision functions are not exposed? |
|
Actually I'm not sure this is part of the pytorch v1.5 release: I can't see the file you're pointing at in the release/1.5 branch and I wasn't able to find it somewhere else in the tree. |
Sorry for the noise! I was under the impression that it was in 1.5, since it was merged before the branch-off, but it seems that it was backed out for minor changed and then merged again to master after the branch point. So it probably won't make it into 1.5.0 :(. Guess we'll have to wait some more time (a colleague is training the same transformers with PyTorch + apex and it's approximately twice as fast on some of our GPUs). |
|
No worries, feel free to remind me to include this once we reach 1.6 if I forget to do so. |
|
Yes, sorry for the noise, but full AMP support didn't make 1.5. There are some building blocks there, but we are expecting full support in 1.6. |
Awesome 👍 I'll try a nightly in the meanwhile! |
This updates tch-rs to using PyTorch 1.5. As of 2020-04-10, PyTorch 1.5 has not been officially released (some release candidates have been), so this PR will only be merged once this has happened.