Bedrock fix

sagemaker-serve/example_notebooks/bedrock_nova_deployment.ipynb

@@ -0,0 +1,210 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Deploy a Fine-Tuned Nova Model to Amazon Bedrock\n",
+    "\n",
+    "This notebook demonstrates how to deploy a fine-tuned Amazon Nova model to\n",
+    "Amazon Bedrock using `BedrockModelBuilder`.\n",
+    "\n",
+    "The workflow:\n",
+    "1. Retrieve a completed Nova SFT training job\n",
+    "2. Create a `BedrockModelBuilder` from the training job\n",
+    "3. Deploy to Bedrock — the builder automatically:\n",
+    "   - Detects the model as Nova\n",
+    "   - Reads the checkpoint URI from the training job manifest\n",
+    "   - Calls `CreateCustomModel` and polls until Active\n",
+    "   - Calls `CreateCustomModelDeployment` and polls until Active\n",
+    "4. Test inference\n",
+    "5. Clean up resources\n",
+    "\n",
+    "**Prerequisites:**\n",
+    "- AWS credentials with SageMaker and Bedrock access\n",
+    "- `sagemaker-serve` package installed\n",
+    "- A completed Nova SFT training job\n",
+    "- An IAM role with Bedrock and SageMaker permissions"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os, json, time, random, boto3\n",
+    "\n",
+    "REGION = \"us-east-1\"\n",
+    "os.environ[\"AWS_DEFAULT_REGION\"] = REGION\n",
+    "os.environ[\"SAGEMAKER_REGION\"] = REGION\n",
+    "\n",
+    "from sagemaker.core.helper.session_helper import get_execution_role\n",
+    "role_arn = get_execution_role()\n",
+    "print(f\"Role: {role_arn}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1: Retrieve the completed training job\n",
+    "\n",
+    "Use an existing completed Nova SFT training job. Replace the job name with your own."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sagemaker.core.resources import TrainingJob\n",
+    "\n",
+    "training_job = TrainingJob.get(training_job_name=\"nova-textgeneration-micro-sft-20251208154822\")\n",
+    "print(f\"Training job: {training_job.training_job_name}\")\n",
+    "print(f\"Status:       {training_job.training_job_status}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2: Deploy to Bedrock with BedrockModelBuilder\n",
+    "\n",
+    "The builder handles the full deployment flow:\n",
+    "- Fetches the model package from the training job\n",
+    "- Detects it as a Nova model\n",
+    "- Reads the checkpoint URI from the training output manifest\n",
+    "- Creates a Bedrock custom model and polls until Active\n",
+    "- Creates a deployment and polls until Active"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sagemaker.serve.bedrock_model_builder import BedrockModelBuilder\n",
+    "\n",
+    "builder = BedrockModelBuilder(model=training_job)\n",
+    "print(f\"Model package: {builder.model_package}\")\n",
+    "print(f\"S3 artifacts:  {builder.s3_model_artifacts}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "rand = random.randint(1000, 9999)\n",
+    "custom_model_name = f\"nova-e2e-{rand}-{int(time.time())}\"\n",
+    "deployment_name = f\"{custom_model_name}-dep\"\n",
+    "\n",
+    "print(f\"Deploying as: {custom_model_name}\")\n",
+    "print(\"This will poll for model creation and deployment — may take several minutes...\")\n",
+    "\n",
+    "response = builder.deploy(\n",
+    "    custom_model_name=custom_model_name,\n",
+    "    role_arn=role_arn,\n",
+    "    deployment_name=deployment_name,\n",
+    ")\n",
+    "\n",
+    "deployment_arn = response.get(\"customModelDeploymentArn\")\n",
+    "print(f\"\\nDeployment ARN: {deployment_arn}\")\n",
+    "print(\"Deployment is Active and ready for inference.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 3: Test inference\n",
+    "\n",
+    "Once the deployment is Active, invoke it via the Bedrock Runtime API.\n",
+    "Nova expects `content` as an array of objects with a `text` key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "bedrock_runtime = boto3.client(\"bedrock-runtime\", region_name=REGION)\n",
+    "\n",
+    "resp = bedrock_runtime.invoke_model(\n",
+    "    modelId=deployment_arn,\n",
+    "    contentType=\"application/json\",\n",
+    "    body=json.dumps({\n",
+    "        \"messages\": [{\"role\": \"user\", \"content\": [{\"text\": \"What is 7 + 7?\"}]}]\n",
+    "    }),\n",
+    ")\n",
+    "\n",
+    "result = json.loads(resp[\"body\"].read())\n",
+    "print(f\"Response: {json.dumps(result, indent=2)}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 4: Cleanup\n",
+    "\n",
+    "Delete the deployment and custom model to avoid ongoing charges."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "bedrock = boto3.client(\"bedrock\", region_name=REGION)\n",
+    "\n",
+    "dep_info = bedrock.get_custom_model_deployment(\n",
+    "    customModelDeploymentIdentifier=deployment_arn\n",
+    ")\n",
+    "model_arn = dep_info.get(\"modelArn\")\n",
+    "\n",
+    "# Delete deployment first\n",
+    "try:\n",
+    "    bedrock.delete_custom_model_deployment(\n",
+    "        customModelDeploymentIdentifier=deployment_arn\n",
+    "    )\n",
+    "    print(f\"Deleted deployment: {deployment_arn}\")\n",
+    "except Exception as e:\n",
+    "    print(f\"Failed to delete deployment: {e}\")\n",
+    "\n",
+    "# Then delete the custom model\n",
+    "try:\n",
+    "    bedrock.delete_custom_model(modelIdentifier=model_arn)\n",
+    "    print(f\"Deleted custom model: {model_arn}\")\n",
+    "except Exception as e:\n",
+    "    print(f\"Failed to delete custom model: {e}\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.12.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}