-
Notifications
You must be signed in to change notification settings - Fork 50
Accept container family as input for model creation and param validation #862
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Accept container family as input for model creation and param validation #862
Conversation
ads/aqua/model/model.py
Outdated
) | ||
|
||
if not inference_container: | ||
inference_container = InferenceContainerTypeKey.AQUA_TGI_CONTAINER_KEY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be based on the check if this model has text_generation_inference tag. If not, we should default to vllm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated this to get the right image family based on the text_generation_inference tag. I thought all text-generation models will have the tag anyway, so didn't add an additional check here.
@@ -40,6 +40,25 @@ class Tags(str, metaclass=ExtendedEnumMeta): | |||
AQUA_EVALUATION_MODEL_ID = "evaluation_model_id" | |||
|
|||
|
|||
class InferenceContainerType(str, metaclass=ExtendedEnumMeta): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fyi - moved this from datasciencemodeldeployment.enums to here to avoid circular imports.
) | ||
logger.debug(message) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: shouldn't we log here the actual error, not the one that we generated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not for this one. We want this to be a user friendly message when this entry is not available in custom metadata.
Description
This PR includes the changes for:
odsc-tgi-serving
.Unit Tests