Changed multi model metadata storage. #1112
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lu-ohai
requested review from
VipulMascarenhas,
ahosler,
darenr,
mayoor,
mrDzurb and
qiuosier
as code owners
oracle-contributor-agreement
bot
added
the
OCA Verified
…racle/accelerated-data-science into ODSC-68579/improved_multi_model_metadata_storage_2
mrDzurb
reviewed
Mar 12, 2025
ads/aqua/constants.py
Outdated
| # Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/ | ||
| """This module defines constants used in ads.aqua module.""" | ||
|
|
||
| UNKNOWN = "" |
There was a problem hiding this comment.
Let's use it form common.utils instead
mrDzurb
reviewed
Mar 12, 2025
| ModelCustomMetadataItem(key=ModelCustomMetadataFields.MULTIMODEL_METADATA), | ||
| ).value | ||
|
|
||
| if not multi_model_metadata_value: |
There was a problem hiding this comment.
error_message = (
f"Required model metadata is missing for evaluation source ID: {create_aqua_evaluation_details.evaluation_source_id}. "
f"A valid multi-model deployment requires {ModelCustomMetadataFields.MULTIMODEL_METADATA}. "
"Please recreate the model deployment and retry the evaluation, as an issue occurred during the initialization of the model group."
)
logger.debug(error_message)
raise AquaRuntimeError(error_message)
ads/aqua/evaluation/evaluation.py
Outdated
| logger.debug(error_message) | ||
| raise AquaRuntimeError(error_message) | ||
|
|
||
| multi_model_metadata = json.loads( |
There was a problem hiding this comment.
Let's add more debug statements, indicating the steps we are doing here.
I also think that we should put the loading the metadata into try/catch to prevent returning useless error messages to the users. For users we should say that something wrong with loading the metadata, and put details into the debug logs.
| ) | ||
|
|
||
| # Build the list of valid model names from custom metadata. | ||
| model_names = [ |
There was a problem hiding this comment.
We should load metadata back to the AquaMultiModelRef
| # Finalize creation | ||
| custom_model.create(model_by_reference=True) | ||
|
|
||
| # Create custom metadata for multi model metadata |
There was a problem hiding this comment.
let's add more debug statements here
| ), | ||
| ).value | ||
| if not multi_model_metadata_value: | ||
| raise AquaRuntimeError( |
There was a problem hiding this comment.
raise AquaRuntimeError(
f"Invalid multi-model deployment: {model_deployment_id}. "
f"Ensure that the required custom metadata `{ModelCustomMetadataFields.MULTIMODEL_METADATA}` is added to the AQUA multi-model `{aqua_model.display_name}` ({aqua_model.id})."
)
mrDzurb
approved these changes
Mar 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improved multi model metadata storage
create_custom_metadata_artifactto store multi model metadata in model catalog while creating the aqua model.get_custom_metadata_artifactto get multi model metadata in model catalog while fetching the aqua deployment.Notebook