Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@ Key Features
* **Abstraction of Low-Level Details**: Automatically handles resource state transitions and polling logic, freeing developers from managing these intricacies and allowing them to focus on higher-level tasks.
* **Auto Code Completion**: Enhances the developer experience by offering real-time suggestions and completions in popular IDEs, reducing syntax errors and speeding up the coding process.
* **Comprehensive Documentation and Type Hints**: Provides detailed guidance and type hints to help developers understand functionalities, write code faster, and reduce errors without complex API navigation.
* **Incorporation of Intelligent Defaults**: Integrates the previous SageMaker SDK feature of intelligent defaults, allowing developers to set default values for parameters like IAM roles and VPC configurations. This streamlines the setup process, enabling developers to focus on customizations specific to their use case.
* **Incorporation of Default Configs**: Integrates the previous SageMaker SDK feature of default configs, allowing developers to set default values for parameters like IAM roles and VPC configurations. This streamlines the setup process, enabling developers to focus on customizations specific to their use case.


Benefits
--------

* **Simplified Development**: By abstracting low-level details and providing intelligent defaults, developers can focus on building and deploying machine learning models without getting bogged down by repetitive tasks.
* **Simplified Development**: By abstracting low-level details and providing default configs, developers can focus on building and deploying machine learning models without getting bogged down by repetitive tasks.
* **Increased Productivity**: The SDK's features, such as auto code completion and type hints, help developers write code faster and with fewer errors.
* **Enhanced Readability**: Resource chaining and dedicated resource classes result in more readable and maintainable code.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Intelligent Defaults and Logging Configuration in SageMakerCore"
"# Default Configs and Logging Configuration in SageMakerCore"
]
},
{
Expand All @@ -14,21 +14,21 @@
"---\n",
"## Introductions\n",
"\n",
"In this notebook, we will walkthrough the setup and usage of intelligent defaults in the SageMakerCore SDK. Additionally, this notebook contains a section with the steps required for configuring logging levels to assist in the debugging of issues that arise while using the SDK.\n",
"In this notebook, we will walkthrough the setup and usage of default configs in the SageMakerCore SDK. Additionally, this notebook contains a section with the steps required for configuring logging levels to assist in the debugging of issues that arise while using the SDK.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Intelligent Defaults\n",
"### Default Configs\n",
"\n",
"Intelligent Defaults is a feature provided by the SageMakerCore SDK to assist users in defining default values to be auto populated into the AWS API Request parameters. For example, if a user/admin wants all of their AWS Resources to use a specific VPC Config during creation this can be defined in the Intelligent Defaults Configs. Intelligent Defaults supports:\n",
"Default Configs is a feature provided by the SageMakerCore SDK to assist users in defining default values to be auto populated into the AWS API Request parameters. For example, if a user/admin wants all of their AWS Resources to use a specific VPC Config during creation this can be defined in the Default Configs Configs. Default Configs supports:\n",
"1. GlobalDefaults - default values applied across SageMaker API calls\n",
"2. Resource Specific Defaults - defaults applied only when creating a specific resource\n",
"\n",
"An Example of the strucuture of the Intelligent Defaults Config is below:"
"An Example of the strucuture of the Default Configs Config is below:"
]
},
{
Expand Down Expand Up @@ -280,16 +280,16 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Intelligent Defaults"
"## Default Configs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create Intelligent Defaults JSON\n",
"### Create Default Configs JSON\n",
"\n",
"In order for SageMakerCore to pick up the Intelligent Defaults Configs to populate API calls, we first must create the json config file and set the `SAGEMAKER_CORE_ADMIN_CONFIG_OVERRIDE` enviornment variable.\n",
"In order for SageMakerCore to pick up the Default Configs Configs to populate API calls, we first must create the json config file and set the `SAGEMAKER_CORE_ADMIN_CONFIG_OVERRIDE` enviornment variable.\n",
"\n",
"Below we will create the config file at `data/defaults.json` and assign this path to the `SAGEMAKER_CORE_ADMIN_CONFIG_OVERRIDE` enviornment variable."
]
Expand Down Expand Up @@ -367,7 +367,7 @@
" \n",
"cluster_name_v3 = 'xgboost-cluster-' + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
"\n",
"# Use vpc_config from Intelligent Defaults JSON config file under the SageMaker.PythonSDK.Resources.GlobalDefaults key\n",
"# Use vpc_config from Default Configs JSON config file under the SageMaker.PythonSDK.Resources.GlobalDefaults key\n",
"cluster = Cluster.create(\n",
" cluster_name=cluster_name_v3,\n",
" instance_groups=[\n",
Expand Down Expand Up @@ -413,8 +413,8 @@
"\n",
"job_name_v3 = 'xgboost-iris-' + time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
"\n",
"# Use role and output_data_config from Intelligent Defaults JSON config file under the SageMaker.PythonSDK.Resources.TrainingJob key\n",
"# Use vpc_config from Intelligent Defaults JSON config file under the SageMaker.PythonSDK.Resources.GlobalDefaults key\n",
"# Use role and output_data_config from Default Configs JSON config file under the SageMaker.PythonSDK.Resources.TrainingJob key\n",
"# Use vpc_config from Default Configs JSON config file under the SageMaker.PythonSDK.Resources.GlobalDefaults key\n",
"\n",
"training_job = TrainingJob.create(\n",
" training_job_name=job_name_v3,\n",
Expand Down
8 changes: 4 additions & 4 deletions integ/test_codegen.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,11 +153,11 @@ def test_training_and_inference(self):
)
endpoint.wait_for_status("InService")

def test_intelligent_defaults(self):
def test_default_configs(self):
os.environ["SAGEMAKER_CORE_ADMIN_CONFIG_OVERRIDE"] = (
self._setup_intelligent_default_configs_and_fetch_path()
self._setup_default_configs_and_fetch_path()
)
job_name_v3 = "xgboost-test-intelligent-default-" + time.strftime(
job_name_v3 = "xgboost-test-default-configs-" + time.strftime(
"%Y-%m-%d-%H-%M-%S", time.gmtime()
)

Expand Down Expand Up @@ -207,7 +207,7 @@ def test_intelligent_defaults(self):
def tearDown(self) -> None:
handle_cleanup()

def _setup_intelligent_default_configs_and_fetch_path(self) -> str:
def _setup_default_configs_and_fetch_path(self) -> str:
DEFAULTS_CONTENT = {
"SchemaVesion": "1.0",
"SageMaker": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from sagemaker_core.main.exceptions import (
LocalConfigNotFoundError,
S3ConfigNotFoundError,
IntelligentDefaultsError,
DefaultConfigsError,
ConfigSchemaValidationError,
)
from sagemaker_core.main.utils import get_textual_rich_logger
Expand Down Expand Up @@ -116,7 +116,7 @@ def _load_config_from_s3(s3_uri, s3_resource_for_config) -> dict:
boto_session = boto3.DEFAULT_SESSION or boto3.Session()
boto_region_name = boto_session.region_name
if boto_region_name is None:
raise IntelligentDefaultsError(
raise DefaultConfigsError(
message=(
"Valid region is not provided in the Boto3 session."
+ "Setup local AWS configuration with a valid region supported by SageMaker."
Expand Down
16 changes: 8 additions & 8 deletions src/sagemaker_core/main/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,21 +91,21 @@ def __init__(self, resource_type="(Unkown)", status="(Unkown)", reason="(Unkown)
super().__init__(resource_type=resource_type, status=status, reason=reason)


### Intelligent Defaults Errors
class IntelligentDefaultsError(SageMakerCoreError):
"""Raised when an error occurs in the Intelligent Defaults"""
### Default Configs Errors
class DefaultConfigsError(SageMakerCoreError):
"""Raised when an error occurs in the Default Configs"""

fmt = "An error occurred while loading Intelligent Default. {message}"
fmt = "An error occurred while loading Default Configs. {message}"

def __init__(self, message="", **kwargs):
"""Initialize an IntelligentDefaultsError exception.
"""Initialize an DefaultConfigsError exception.
Args:
message (str): A message describing the error.
"""
super().__init__(message=message, **kwargs)


class LocalConfigNotFoundError(IntelligentDefaultsError):
class LocalConfigNotFoundError(DefaultConfigsError):
"""Raised when a configuration file is not found in local file system"""

fmt = "Failed to load configuration file from location: {file_path}. {message}"
Expand All @@ -119,7 +119,7 @@ def __init__(self, file_path="(Unkown)", message=""):
super().__init__(file_path=file_path, message=message)


class S3ConfigNotFoundError(IntelligentDefaultsError):
class S3ConfigNotFoundError(DefaultConfigsError):
"""Raised when a configuration file is not found in S3"""

fmt = "Failed to load configuration file from S3 location: {s3_uri}. {message}"
Expand All @@ -133,7 +133,7 @@ def __init__(self, s3_uri="(Unkown)", message=""):
super().__init__(s3_uri=s3_uri, message=message)


class ConfigSchemaValidationError(IntelligentDefaultsError, ValidationError):
class ConfigSchemaValidationError(DefaultConfigsError, ValidationError):
"""Raised when a configuration file does not adhere to the schema"""

fmt = "Failed to validate configuration file from location: {file_path}. {message}"
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker_core/main/resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
is_primitive_list,
serialize,
)
from sagemaker_core.main.intelligent_defaults_helper import (
from sagemaker_core.main.default_configs_helper import (
load_default_configs_for_resource_name,
get_config_value,
)
Expand Down
2 changes: 1 addition & 1 deletion src/sagemaker_core/tools/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@

GENERATED_CLASSES_LOCATION = os.getcwd() + "/src/sagemaker_core/main"
UTILS_CODEGEN_FILE_NAME = "utils.py"
INTELLIGENT_DEFAULTS_HELPER_CODEGEN_FILE_NAME = "intelligent_defaults_helper.py"
DEFAULT_CONFIGS_CODEGEN_FILE_NAME = "default_configs_helper.py"

RESOURCES_CODEGEN_FILE_NAME = "resources.py"

Expand Down
14 changes: 7 additions & 7 deletions src/sagemaker_core/tools/resources_codegen.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
import json
from sagemaker_core.main.code_injection.codec import pascal_to_snake
from sagemaker_core.main.config_schema import SAGEMAKER_PYTHON_SDK_CONFIG_SCHEMA
from sagemaker_core.main.exceptions import IntelligentDefaultsError
from sagemaker_core.main.exceptions import DefaultConfigsError
from sagemaker_core.main.utils import get_textual_rich_logger
from sagemaker_core.tools.constants import (
BASIC_RETURN_TYPES,
Expand Down Expand Up @@ -190,7 +190,7 @@ def generate_imports(self) -> str:
"from sagemaker_core.main.code_injection.constants import Color",
"from sagemaker_core.main.utils import SageMakerClient, ResourceIterator, Unassigned, get_textual_rich_logger, "
"snake_to_pascal, pascal_to_snake, is_not_primitive, is_not_str_dict, is_primitive_list, serialize",
"from sagemaker_core.main.intelligent_defaults_helper import load_default_configs_for_resource_name, get_config_value",
"from sagemaker_core.main.default_configs_helper import load_default_configs_for_resource_name, get_config_value",
"from sagemaker_core.main.logs import MultiLogStreamHandler",
"from sagemaker_core.main.exceptions import *",
"import sagemaker_core.main.shapes as shapes",
Expand Down Expand Up @@ -859,7 +859,7 @@ def generate_create_method(self, resource_name: str, **kwargs) -> str:
operation_input_shape_name=operation_input_shape_name,
include_session_region=True,
include_return_resource_docstring=True,
include_intelligent_defaults_errors=True,
include_default_configs_errors=True,
)

if "Describe" + resource_name in self.operations:
Expand Down Expand Up @@ -956,7 +956,7 @@ def _generate_docstring(
include_session_region: bool = False,
include_return_resource_docstring: bool = False,
return_string: str = None,
include_intelligent_defaults_errors: bool = False,
include_default_configs_errors: bool = False,
exclude_resource_attrs: list = None,
) -> str:
"""
Expand All @@ -970,7 +970,7 @@ def _generate_docstring(
include_session_region (bool): Whether to include session and region documentation.
include_return_resource_docstring (bool): Whether to include resource-specific documentation.
return_string (str): The return string.
include_intelligent_defaults_errors (bool): Whether to include intelligent defaults errors.
include_default_configs_errors (bool): Whether to include default configs errors.
exclude_resource_attrs (list): A list of attributes to exclude from the docstring.

Returns:
Expand Down Expand Up @@ -1000,8 +1000,8 @@ def _generate_docstring(

docstring += self._exception_docstring(operation_name)

if include_intelligent_defaults_errors:
subclasses = set(IntelligentDefaultsError.__subclasses__())
if include_default_configs_errors:
subclasses = set(DefaultConfigsError.__subclasses__())
_id_exception_docstrings = [
f"\n {subclass.__name__}: {subclass.__doc__}" for subclass in subclasses
]
Expand Down