Skip to content

Conversation

@vishalbollu
Copy link
Contributor

@vishalbollu vishalbollu commented Mar 8, 2021

Torch 1.8.0 came out recently and certain operations don't seem to be supported for CUDA 10.2 cudnn 7/8 on a Tesla T4. It appears to not work for cuda 10.1 cudnn 7 as well.

Torch 1.7 seems to be working as expected. Temporarily set the Torch version to 1.7 till issues on Torch 1.8 are resolved. Here is a related issue: pytorch/pytorch#53336.


checklist:

  • run make test and make lint
  • test manually (i.e. build/push all images, restart operator, and re-deploy APIs)

@vishalbollu vishalbollu requested a review from RobertLucian March 8, 2021 23:11
@vishalbollu vishalbollu changed the title Temporarily lock torch version 1.7 for pytorch text-generator gpu tests Temporarily lock torch to 1.7 for pytorch text-generator gpu tests Mar 8, 2021
Copy link
Member

@RobertLucian RobertLucian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vishalbollu vishalbollu merged commit b0e7412 into master Mar 8, 2021
@vishalbollu vishalbollu deleted the lock-torch-version-1.7 branch March 8, 2021 23:36
@vishalbollu vishalbollu added this to the 0.31 milestone Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants