notebooks/protocol_examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Examples with Different Protocols\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    " * A kubernetes cluster with kubectl configured\n",
    " * curl\n",
    " * grpcurl\n",
    " * pygmentize\n",
    " \n",
    "## Examples\n",
    "\n",
    "  * [Open Inference Protocol or V2 Protocol](#V2-Protocol-Model)\n",
    "  * [Seldon Protocol](#Seldon-Protocol-Model)\n",
    "  * [Tensorflow Protocol](#Tensorflow-Protocol-Model)\n",
    "\n",
    "**Note**:Seldon has adopted the industry-standard Open Inference Protocol (OIP) and is no longer maintaining the Seldon and TensorFlow protocols. This transition allows for greater interoperability among various model serving runtimes, such as MLServer. To learn more about implementing OIP for model serving in Seldon Core 1, see [MLServer](https://docs.seldon.ai/mlserver).\n",
    "\n",
    "We strongly encourage you to adopt the OIP, which provides seamless integration across diverse model serving runtimes, supports the development of versatile client and benchmarking tools, and ensures a high-performance, consistent, and unified inference experience.\n",
    "\n",
    "## Setup Seldon Core\n",
    "\n",
    "Use the setup notebook to [Setup Cluster](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html) to setup Seldon Core with an ingress - either Ambassador or Istio.\n",
    "\n",
    "Then port-forward to that ingress on localhost:8003 in a separate terminal either with:\n",
    "\n",
    " * Ambassador: `kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080`\n",
    " * Istio: `kubectl port-forward $(kubectl get pods -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system 8003:8080`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "0764c40b-d635-4fa6-9850-e04cb32fd069"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error from server (AlreadyExists): namespaces \"seldon\" already exists\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "85bfd753-1ecd-44ab-9d6a-357b348bd2b2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"kind-ansible\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "b35405d9-bf57-496c-aa86-888679ab81dc"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "d2a9a0cf-d605-441f-9eef-23c1cbafa442"
   },
   "outputs": [],
   "source": [
    "from IPython.core.magic import register_line_cell_magic\n",
    "\n",
    "@register_line_cell_magic\n",
    "def writetemplate(line, cell):\n",
    "    with open(line, 'w') as f:\n",
    "        f.write(cell.format(**globals()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "22f1587b-0167-4e14-91fd-08046f4c25ff"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.17.0-dev'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "VERSION=!cat ../version.txt\n",
    "VERSION=VERSION[0]\n",
    "VERSION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Seldon Protocol Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will deploy a REST model that uses the SELDON Protocol namely by specifying the attribute `protocol: seldon`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "66dee291-bb5c-403a-8e6b-687572297d02"
   },
   "outputs": [],
   "source": [
    "%%writetemplate resources/model_seldon.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: example-seldon\n",
    "spec:\n",
    "  protocol: seldon\n",
    "  predictors:\n",
    "  - componentSpecs:\n",
    "    - spec:\n",
    "        containers:\n",
    "        - image: seldonio/mock_classifier:{VERSION}\n",
    "          name: classifier\n",
    "    graph:\n",
    "      name: classifier\n",
    "      type: MODEL\n",
    "    name: model\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "f8cbfcc3-fa4b-44e6-83df-1c6a7e88f2d4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model_seldon.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "72be8b12-3e5e-4834-8cab-70acc6c877fe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-seldon condition met\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "c3345e2b-59fb-41fd-802e-355746eed581",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': ['proba'], 'ndarray': [[0.43782349911420193]]}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.17.0-dev'}}}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/example-seldon/api/v1.0/predictions \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)\n",
    "assert(d[\"data\"][\"ndarray\"][0][0] > 0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "ca49cda7-6ca1-4b65-90a5-46cf3ea2517e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.17.0-dev'}}, 'data': {'names': ['proba'], 'ndarray': [[0.43782349911420193]]}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl -d '{\"data\":{\"ndarray\":[[1.0,2.0,5.0]]}}' \\\n",
    "         -rpc-header seldon:example-seldon -rpc-header namespace:seldon \\\n",
    "         -plaintext \\\n",
    "         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)\n",
    "assert(d[\"data\"][\"ndarray\"][0][0] > 0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "289877a5-3e9d-412d-8e8a-899788d23546"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"example-seldon\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model_seldon.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Seldon protocol Model with ModelUri with two custom models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "dfec102d-fb3d-411b-9278-6b13664d29cd"
   },
   "outputs": [],
   "source": [
    "%%writetemplate resources/model_seldon.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: example-seldon\n",
    "spec:\n",
    "  protocol: seldon\n",
    "  predictors:\n",
    "  - componentSpecs:\n",
    "    - spec:\n",
    "        containers:\n",
    "        - image: seldonio/mock_classifier:{VERSION}\n",
    "          name: classifier\n",
    "        - image: seldonio/mock_classifier:{VERSION}\n",
    "          name: classifier2\n",
    "    graph:\n",
    "      name: classifier\n",
    "      type: MODEL\n",
    "      modelUri: gs://seldon-models/v1.17.0-dev/sklearn/iris\n",
    "      children:\n",
    "      - name: classifier2\n",
    "        type: MODEL\n",
    "        modelUri: gs://seldon-models/v1.17.0-dev/sklearn/iris\n",
    "    name: model\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "72680c5d-b32d-48ea-b1ce-f759e31a76d2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-seldon unchanged\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model_seldon.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "a345f65d-6b9b-4035-952b-99b567a18f3f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-seldon condition met\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "124572ac-9dda-4988-9acf-6bff6721fd6e",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': ['proba'], 'ndarray': [[0.07735472603574542]]}, 'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.17.0-dev', 'classifier2': 'seldonio/mock_classifier:1.17.0-dev'}}}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0]]}}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/example-seldon/api/v1.0/predictions \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "a46120ee-9254-4ddf-9135-43e5f0f8c5cd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {'requestPath': {'classifier': 'seldonio/mock_classifier:1.17.0-dev', 'classifier2': 'seldonio/mock_classifier:1.17.0-dev'}}, 'data': {'names': ['proba'], 'ndarray': [[0.07735472603574542]]}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl -d '{\"data\":{\"ndarray\":[[1.0,2.0,5.0]]}}' \\\n",
    "         -rpc-header seldon:example-seldon -rpc-header namespace:seldon \\\n",
    "         -plaintext \\\n",
    "         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "849c60f7-ee6c-47a0-ab8e-b4b7473d2c02"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"example-seldon\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model_seldon.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensorflow Protocol Model\n",
    "We will deploy a model that uses the TENSORLFOW Protocol namely by specifying the attribute `protocol: tensorflow`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "3fc819f1-c08f-439e-bb4a-7f25486876ee"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing resources/model_tfserving.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile resources/model_tfserving.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: example-tfserving\n",
    "spec:\n",
    "  protocol: tensorflow\n",
    "  predictors:\n",
    "  - componentSpecs:\n",
    "    - spec:\n",
    "        containers:\n",
    "        - args: \n",
    "          - --port=8500\n",
    "          - --rest_api_port=8501\n",
    "          - --model_name=halfplustwo\n",
    "          - --model_base_path=gs://seldon-models/tfserving/half_plus_two\n",
    "          image: tensorflow/serving\n",
    "          name: halfplustwo\n",
    "          ports:\n",
    "          - containerPort: 8501\n",
    "            name: http\n",
    "            protocol: TCP\n",
    "          - containerPort: 8500\n",
    "            name: grpc\n",
    "            protocol: TCP\n",
    "    graph:\n",
    "      name: halfplustwo\n",
    "      type: MODEL\n",
    "      endpoint:\n",
    "        httpPort: 8501\n",
    "        grpcPort: 8500\n",
    "    name: model\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "8a4fdf7e-a6cf-43e4-b3b0-4a78c59d6a77"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-tfserving created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model_tfserving.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "902a32de-4e41-497b-954b-b3c6179e5b61"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/example-tfserving condition met\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "febb95e9-a0e7-427a-97df-810e2ec1809a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'predictions': [2.5, 3.0, 4.5]}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"instances\": [1.0, 2.0, 5.0]}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/example-tfserving/v1/models/halfplustwo/:predict \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)\n",
    "assert(d[\"predictions\"][0] == 2.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "99d5d9e2-6668-4581-a467-e575cdb60ddd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'outputs': {'x': {'dtype': 'DT_FLOAT', 'tensorShape': {'dim': [{'size': '3'}]}, 'floatVal': [2.5, 3, 3.5]}}, 'modelSpec': {'name': 'halfplustwo', 'version': '123', 'signatureName': 'serving_default'}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl \\\n",
    "   -d '{\"model_spec\":{\"name\":\"halfplustwo\"},\"inputs\":{\"x\":{\"dtype\": 1, \"tensor_shape\": {\"dim\":[{\"size\": 3}]}, \"floatVal\" : [1.0, 2.0, 3.0]}}}' \\\n",
    "   -rpc-header seldon:example-tfserving -rpc-header namespace:seldon \\\n",
    "   -plaintext -proto ./prediction_service.proto \\\n",
    "   0.0.0.0:8003 tensorflow.serving.PredictionService/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)\n",
    "assert(d[\"outputs\"][\"x\"][\"floatVal\"][0] == 2.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "8537fd9f-48f2-45a1-a9a7-01fe6038a7fd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"example-tfserving\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model_tfserving.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V2 Protocol Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will deploy a REST model that uses the V2 Protocol namely by specifying the attribute `protocol: v2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "34fb9927-0b02-416f-a810-a3f6a9dfa666"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing resources/model_v2.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile resources/model_v2.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: triton\n",
    "spec:\n",
    "  protocol: v2\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: TRITON_SERVER\n",
    "      modelUri: gs://seldon-models/trtis/simple-model\n",
    "      name: simple\n",
    "    name: simple\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "e6a78666-3300-465c-b79b-d1815f13ed1f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/triton created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model_v2.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "7951851b-8fd1-4aa4-86e6-df70cf8ac653"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/triton condition met\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl wait --for condition=ready --timeout=300s sdep --all -n seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "8a86f8d8-6463-402b-bf48-a39fad95a60d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'model_name': 'simple', 'model_version': '1', 'outputs': [{'name': 'OUTPUT0', 'datatype': 'INT32', 'shape': [1, 16], 'data': [2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32]}, {'name': 'OUTPUT1', 'datatype': 'INT32', 'shape': [1, 16], 'data': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}]}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"inputs\":[{\"name\":\"INPUT0\",\"data\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],\"datatype\":\"INT32\",\"shape\":[1,16]},{\"name\":\"INPUT1\",\"data\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],\"datatype\":\"INT32\",\"shape\":[1,16]}]}'  \\\n",
    "        -X POST http://0.0.0.0:8003/seldon/seldon/triton/v2/models/simple/infer \\\n",
    "        -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)\n",
    "assert(d[\"outputs\"][0][\"data\"][0]==2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "35663143-60ac-4404-9da8-01affe3fc8eb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{  \"modelName\": \"simple\",  \"modelVersion\": \"1\",  \"outputs\": [    {      \"name\": \"OUTPUT0\",      \"datatype\": \"INT32\",      \"shape\": [        \"1\",        \"16\"      ]    },    {      \"name\": \"OUTPUT1\",      \"datatype\": \"INT32\",      \"shape\": [        \"1\",        \"16\"      ]    }  ],  \"rawOutputContents\": [    \"AgAAAAQAAAAGAAAACAAAAAoAAAAMAAAADgAAABAAAAASAAAAFAAAABYAAAAYAAAAGgAAABwAAAAeAAAAIAAAAA==\",    \"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\"  ]}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/api/grpc/kfserving/inference && \\\n",
    "        grpcurl -d '{\"model_name\":\"simple\",\"inputs\":[{\"name\":\"INPUT0\",\"contents\":{\"int_contents\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]},\"datatype\":\"INT32\",\"shape\":[1,16]},{\"name\":\"INPUT1\",\"contents\":{\"int_contents\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]},\"datatype\":\"INT32\",\"shape\":[1,16]}]}' \\\n",
    "        -plaintext -proto ./grpc_service.proto \\\n",
    "        -rpc-header seldon:triton -rpc-header namespace:seldon \\\n",
    "        0.0.0.0:8003 inference.GRPCInferenceService/ModelInfer\n",
    "X=\"\".join(X)\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "5d1fee1a-17ed-4c29-afb5-9224b8ed1213"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"triton\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model_v2.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "078e5944-d29b-46db-926b-533e6c439071"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"example-tfserving\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model_tfserving.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "b48402df-bf93-4df3-a6ae-bf96d7d40b30"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}