Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when trying MII with dtype=fp32 with sampling #101

Closed
marshmellow77 opened this issue Nov 17, 2022 · 2 comments
Closed

Error when trying MII with dtype=fp32 with sampling #101

marshmellow77 opened this issue Nov 17, 2022 · 2 comments

Comments

@marshmellow77
Copy link

When running the example from https://github.com/microsoft/deepspeed-mii#deploying-mii-public in fp32 I receive an AioRpcError error. See this notebook for a minimal example to reproduce the error.

  • 1 T4, GPU memory 16GB
  • deepspeed-mii version 0.0.3
  • transformers version 4.24.0
  • Amazon Linux 2
@marshmellow77 marshmellow77 mentioned this issue Nov 17, 2022
Closed
@mrwyattii
Copy link
Contributor

please see #102 (comment)

@marshmellow77
Copy link
Author

Makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mrwyattii @marshmellow77