-
Notifications
You must be signed in to change notification settings - Fork 57
[AQUA] Add Hugging Face model support to Shape Recommender #1262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
1 similar comment
ads/aqua/shaperecommend/constants.py
Outdated
"UNKNOWN_ENUM_VALUE": "N/A", | ||
} | ||
|
||
HUGGINGFACE_CONFIG_URL = "https://huggingface.co/{model_id}/resolve/main/config.json" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it is not used anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ads/aqua/shaperecommend/recommend.py
Outdated
# Copyright (c) 2025 Oracle and/or its affiliates. | ||
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/ | ||
|
||
#!/usr/bin/env python |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to duplicate this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ads/aqua/shaperecommend/recommend.py
Outdated
logger.info( | ||
f"'{model_id}' is not an OCID, treating as a Hugging Face model ID." | ||
) | ||
# if not compartment_id: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let’s copy the commented-out code somewhere else to keep the main code clean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ads/aqua/shaperecommend/recommend.py
Outdated
from typing import List, Union | ||
import os | ||
import json | ||
from typing import List, Union, Optional, Dict, Any, Tuple |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like type - Any is not used anywhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
ads/aqua/shaperecommend/recommend.py
Outdated
|
||
import shutil | ||
from typing import List, Union | ||
import os |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With your formatter, could you sort the imports?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
return shape_recommendation_report | ||
|
||
def valid_compute_shapes(self, compartment_id: str) -> List["ComputeShapeSummary"]: | ||
def _get_model_config_and_name( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: It would be more clean if we had return at the end:
if is_valid_ocid(model_id):
logger.info(f"Detected OCID: Fetching OCI model config for '{model_id}'.")
ds_model = self._validate_model_ocid(model_id)
config = self._get_model_config(ds_model)
model_name = ds_model.display_name
else:
logger.info(f"Assuming Hugging Face model ID: Fetching config for '{model_id}'.")
config = self._fetch_hf_config(model_id)
model_name = model_id
return config, model_name
ads/aqua/shaperecommend/recommend.py
Outdated
return json.load(f) | ||
except HfHubHTTPError as e: | ||
if "401" in str(e): | ||
raise AquaValueError( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think HF CLI should also have some command to register token? If so we cna offer both options in the message.
config_path = hf_hub_download(repo_id=model_id, filename="config.json") | ||
with open(config_path, "r", encoding="utf-8") as f: | ||
return json.load(f) | ||
except HfHubHTTPError as e: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please check the existing - format_hf_custom_error_message
, i'm wondering if it can be reused here?
ads/aqua/shaperecommend/recommend.py
Outdated
environment variables. | ||
""" | ||
if not compartment_id: | ||
compartment_id = os.environ.get( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in ads.config.py
we have a unified - COMPARTMENT_OCID
variable, maybe it an be reused?
ads/aqua/shaperecommend/recommend.py
Outdated
"A compartment OCID is required to list available shapes. " | ||
"Please provide it as a parameter or set the 'NB_SESSION_COMPARTMENT_OCID' " | ||
"or 'PROJECT_COMPARTMENT_OCID' environment variable." | ||
"cli command: export NB_SESSION_COMPARTMENT_OCID=<NB_SESSION_COMPARTMENT_OCID>" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be better to give AQUA CLI command that will include compartment as an input parameter:
ads aqua deployment recommend_shape --model_id "<OCID>" --compartment_id "<OCID>"
The Aqua Shape Recommender now accepts Hugging Face model IDs in addition to OCI model OCIDs and makes the compartment_id parameter optional.
Key changes:
compartment_id
parameter is now optional. If not provided, it will be sourced from environment variables. An error is raised if no compartment ID can be found.ensure
HF_TOKEN
andNB_SESSION_COMPARTMENT_OCID
orPROJECT_COMPARTMENT_OCID
are set as environment variablesaqua recommend --model-id "distilbert-base-uncased"