[WIP- FINAL][AQUA][GPU Shape Recommendation] Support for Service Managed Models #1252 #1267
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elizjo
requested review from
darenr,
mayoor,
mrDzurb,
VipulMascarenhas,
qiuosier and
ahosler
as code owners
oracle-contributor-agreement
bot
added
the
OCA Verified
mrDzurb
previously approved these changes
Sep 12, 2025
|
📌 Cov diff with main: 📌 Overall coverage: |
|
📌 Cov diff with main: 📌 Overall coverage: |
mrDzurb
approved these changes
Sep 16, 2025
VipulMascarenhas
approved these changes
Sep 16, 2025
| If model type is unsupported by tool (no recommendation report generated) | ||
| If the model type is unsupported and no recommendation report can be generated. | ||
| """ | ||
| deployment_config = self.get_deployment_config(model_id=kwargs.get("model_id")) |
There was a problem hiding this comment.
nit: add a validation here to check if kwargs has model id before passing to get_deployment_config.
| """ | ||
| c = self.llm_config | ||
| embedding_count = 1 if getattr(c, "tie_word_embeddings", True) else 2 | ||
| llm_config = self.llm_config |
There was a problem hiding this comment.
nit: we could just do self.llm_config below instead of declaring a new var, just personal preference. We can keep it as is for now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For AQUA Service models, pre-defined configuration files already exist and can be directly used. This PR adds support for service models via obtaining their pre-defined config files, parsing this metadata, and formatting a response consistent with the current output of the GPU Shape Recommendation Tool
Returns
