Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove bf16 from inference config dtye enum #3010

Merged
merged 8 commits into from Mar 22, 2023
Merged

Remove bf16 from inference config dtye enum #3010

merged 8 commits into from Mar 22, 2023

Conversation

molly-smith
Copy link
Contributor

@molly-smith molly-smith commented Mar 13, 2023

Remove bf16 from inference config dtye enum because not it is not supported. Users should now see pydantic error with supported types vs. vague CUDA error.

pydantic.error_wrappers.ValidationError: 1 validation error for DeepSpeedInferenceConfig
dtype
  value is not a valid enumeration member; permitted: torch.float16, torch.float32, torch.int8 (type=type_error.enum; enum_values=[<DtypeEnum.fp16: torch.float16, 'torch.float16', 'fp16', 'float16', 'half'>, <DtypeEnum.fp32: torch.float32, 'torch.float32', 'fp32', 'float32', 'float'>, <DtypeEnum.int8: torch.int8, 'torch.int8', 'int8'>])

#2954

@jeffra jeffra added the merge-queue PRs ready to merge label Mar 17, 2023
@jeffra jeffra merged commit 27e1b02 into master Mar 22, 2023
@jeffra jeffra deleted the mosm/bf16_check branch March 22, 2023 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge-queue PRs ready to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants