generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 80
Open
Description
When deploying a custom model using the CLI and describing the endpoint, the following info are returned (as an example):
hyp describe hyp-custom-endpoint --name my-custom-tinyllama-endpoint
Deployment (should be completed in 1-5 min):
Deployment State: DeploymentInProgress
Metadata Name: my-custom-tinyllama-endpoint
Namespace: default
Label:
Invocation Endpoint invocations
Instance Type ml.g5.8xlarge
Metrics Enabled False
Model Name tinyllama
Model Version
Model Source Type s3
Model Location tinyllama-1.1b-chat/
Prefetch Enabled False
TLS Cert S3 URI s3://<bucket_name>/certificates/
FSx DNS Name
FSx File System ID
FSx Mount Name
S3 Bucket Name <bucket_name>
S3 Region us-west-2
Image URI 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.33.0-lmi15.0.0-cu128
Container Port 8080
Model Volume Mount Path /opt/ml/model
Model Volume Mount Name modelmount
Resources Limits
Resources Requests
Dimensions
Metric Collection Period 300
Metric Collection Start Time 300
Metric Name
Metric Stat Average
Metric Type Average
Min Value 0.0
CW Trigger Name
CW Trigger Namespace
Target Value
Use Cached Metrics True
Deployment Status Conditions:
| TYPE | STATUS | LAST TRANSITION | LAST UPDATE | MESSAGE |
|-------------|----------|----------------------|----------------------|---------------------------------------------------------------------|
| Available | False | 2025-11-10T14:47:41Z | 2025-11-10T14:47:41Z | Deployment does not have minimum availability. |
| Progressing | True | 2025-11-10T14:47:41Z | 2025-11-10T14:47:41Z | ReplicaSet "my-custom-tinyllama-endpoint-c749664f6" is progressing. |
────────────────────────────────────────────────────────────
SageMaker Endpoint (takes ~10 min to create):
<no SageMaker endpoint information available>
Sagemaker Endpoint Status Conditions:
<none>
At the same time there are additional POD events available that would give more detailed info on what is happening:
kubectl events --for pod/my-custom-tinyllama-endpoint-c749664f6-885bq
LAST SEEN TYPE REASON OBJECT MESSAGE
7m27s Normal Scheduled Pod/my-custom-tinyllama-endpoint-c749664f6-885bq Successfully assigned default/my-custom-tinyllama-endpoint-c749664f6-885bq to hyperpod-i-0450d584f672e7135
7m26s Normal Pulling Pod/my-custom-tinyllama-endpoint-c749664f6-885bq Pulling image "763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.33.0-lmi15.0.0-cu128"
4m4s Normal Pulled Pod/my-custom-tinyllama-endpoint-c749664f6-885bq Successfully pulled image "763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.33.0-lmi15.0.0-cu128" in 3m21.752s (3m21.752s including waiting). Image size: 16180253162 bytes.
4m4s Normal Created Pod/my-custom-tinyllama-endpoint-c749664f6-885bq Created container: my-custom-tinyllama-endpoint
4m3s Normal Started Pod/my-custom-tinyllama-endpoint-c749664f6-885bq Started container my-custom-tinyllama-endpoint
It would be good to return such events when describing the endpoint.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels