notebooks/server_examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example Model Servers with Seldon\n",
    "\n",
    "## Setup Seldon Core\n",
    "\n",
    "Use the setup notebook to [Setup Cluster](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Setup-Cluster) with [Ambassador Ingress](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Ambassador) and [Seldon Core](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Install-Seldon-Core). Instructions [also online](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error from server (AlreadyExists): namespaces \"seldon\" already exists\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"kind-kind\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve SKLearn Iris Model\n",
    "\n",
    "In order to deploy SKLearn artifacts, we can leverage the [pre-packaged SKLearn inference server](https://docs.seldon.io/projects/seldon-core/en/latest/servers/sklearn.html).\n",
    "The exposed API can follow either:\n",
    "\n",
    "- The default Seldon protocol. \n",
    "- The [KFServing V2 protocol](https://docs.seldon.io/projects/seldon-core/en/latest/servers/sklearn.html##v2-kfserving-protocol-incubating).\n",
    "\n",
    "For details on each of these protocols, you can check the [documentation section on API protocols](https://docs.seldon.io/projects/seldon-core/en/latest/graph/protocols.html#v2-kfserving-protocol).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Default Seldon protocol\n",
    "\n",
    "To deploy and start serving an SKLearn artifact using Seldon's default protocol, we can use a config like the one below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../servers/sklearnserver/samples/iris.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../servers/sklearnserver/samples/iris.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: sklearn\n",
    "spec:\n",
    "  name: iris\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: SKLEARN_SERVER\n",
    "      modelUri: gs://seldon-models/sklearn/iris\n",
    "      name: classifier\n",
    "    name: default\n",
    "    replicas: 1\n",
    "    svcOrchSpec: \n",
    "      env: \n",
    "      - name: SELDON_LOG_LEVEL\n",
    "        value: DEBUG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then apply it to deploy it to our Kubernetes cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/sklearn created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/sklearnserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"sklearn-default-0-classifier\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=sklearn -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once it's deployed we can send our sklearn model requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': ['t:0', 't:1', 't:2'], 'ndarray': [[9.912315378486718e-07, 0.0007015931307743852, 0.9992974156376878]]}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0, 6.0]]}}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/sklearn/api/v1.0/predictions \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"sklearn\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "meta {\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 4\n",
      "    values: 0.6806334106409295\n",
      "    values: 0.28830360482888695\n",
      "    values: 0.8975654688578534\n",
      "    values: 0.24493223527153596\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "{'data': {'names': ['t:0', 't:1', 't:2'], 'tensor': {'shape': [1, 3], 'values': [0.14449927217770223, 0.5104835080299303, 0.34501721979236755]}}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,4))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1, 4], 'values': [0.22157381246204433, 0.020827790786912548, 0.7633031884959527, 0.19217598516001178]}}}\n",
      "Response:\n",
      "{'meta': {}, 'data': {'names': ['t:0', 't:1', 't:2'], 'tensor': {'shape': [1, 3], 'values': [0.10758420949891327, 0.4796849750770796, 0.4127308154240072]}}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"grpc\",shape=(1,4))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {}, 'data': {'names': ['t:0', 't:1', 't:2'], 'ndarray': [[9.912315378486718e-07, 0.0007015931307743852, 0.9992974156376878]]}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl -d '{\"data\":{\"ndarray\":[[1.0,2.0,5.0,6.0]]}}' \\\n",
    "         -rpc-header seldon:sklearn -rpc-header namespace:seldon \\\n",
    "         -plaintext \\\n",
    "         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And delete the model we deployed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"sklearn\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/sklearnserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KFServing V2 protocol\n",
    "\n",
    "We can deploy a SKLearn artifact, exposing an API compatible with [KFServing's V2 Protocol](https://docs.seldon.io/projects/seldon-core/en/latest/servers/sklearn.html##v2-kfserving-protocol-incubating) by specifying the `protocol` of our `SeldonDeployment` as `kfserving`.\n",
    "For example, we can consider the config below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../testing/resources/iris-sklearn-v2.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../testing/resources/iris-sklearn-v2.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: sklearn\n",
    "spec:\n",
    "  name: iris\n",
    "  protocol: kfserving\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: SKLEARN_SERVER\n",
    "      modelUri: gs://seldon-models/sklearn/iris\n",
    "      name: classifier\n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then apply it to deploy our model to our Kubernetes cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/sklearn created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../testing/resources/iris-sklearn-v2.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"sklearn-default-0-classifier\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=sklearn -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once it's deployed, we can send inference requests to our model.\n",
    "Note that, since it's using the KFServing's V2 Protocol, these requests will be different to the ones using the default Seldon Protocol."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"model_name\": \"classifier\",\n",
      "  \"model_version\": \"v1\",\n",
      "  \"id\": \"7bc06500-ab6e-45fb-8b9b-00ee02cd6fcd\",\n",
      "  \"parameters\": null,\n",
      "  \"outputs\": [\n",
      "    {\n",
      "      \"name\": \"predict\",\n",
      "      \"shape\": [\n",
      "        1\n",
      "      ],\n",
      "      \"datatype\": \"FP32\",\n",
      "      \"parameters\": null,\n",
      "      \"data\": [\n",
      "        2\n",
      "      ]\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "inference_request = {\n",
    "    \"inputs\": [\n",
    "        {\n",
    "          \"name\": \"predict\",\n",
    "          \"shape\": [1, 4],\n",
    "          \"datatype\": \"FP32\",\n",
    "          \"data\": [[1, 2, 3, 4]]\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "endpoint = \"http://localhost:8003/seldon/seldon/sklearn/v2/models/infer\"\n",
    "response = requests.post(endpoint, json=inference_request)\n",
    "\n",
    "print(json.dumps(response.json(), indent=2))\n",
    "assert response.ok"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can delete the model we deployed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"sklearn\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../testing/resources/iris-sklearn-v2.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve XGBoost Iris Model\n",
    "\n",
    "In order to deploy XGBoost models, we can leverage the [pre-packaged XGBoost inference server](https://docs.seldon.io/projects/seldon-core/en/latest/servers/xgboost.html).\n",
    "The exposed API can follow either:\n",
    "\n",
    "- The default Seldon protocol. \n",
    "- The [KFServing V2 protocol](https://docs.seldon.io/projects/seldon-core/en/latest/servers/xgboost.html##v2-kfserving-protocol-incubating).\n",
    "\n",
    "For details on each of these protocols, you can check the [documentation section on API protocols](https://docs.seldon.io/projects/seldon-core/en/latest/graph/protocols.html#v2-kfserving-protocol)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Default Seldon protocol\n",
    "\n",
    "We can deploy a XGBoost model uploaded to an object store by using the XGBoost model server implementation as shown in the config below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../servers/xgboostserver/samples/iris.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../servers/xgboostserver/samples/iris.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: xgboost\n",
    "spec:\n",
    "  name: iris\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: XGBOOST_SERVER\n",
    "      modelUri: gs://seldon-models/xgboost/iris\n",
    "      name: classifier\n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then we apply it to deploy it to our kubernetes cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/xgboost created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/xgboostserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"xgboost-default-0-classifier\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=xgboost -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Rest Requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': [], 'ndarray': [2.0]}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"data\": {\"ndarray\":[[1.0, 2.0, 5.0, 6.0]]}}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/xgboost/api/v1.0/predictions \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"xgboost\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "meta {\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 4\n",
      "    values: 0.9816489970625223\n",
      "    values: 0.15136579121850824\n",
      "    values: 0.9369540193482753\n",
      "    values: 0.6637451146599143\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "{'data': {'names': [], 'tensor': {'shape': [1], 'values': [0.0]}}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,4))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1, 4], 'values': [0.3316136646069059, 0.35708074352350294, 0.7611407638250924, 0.9509923468248295]}}}\n",
      "Response:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1], 'values': [0.0]}}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"grpc\",shape=(1,4))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {}, 'data': {'ndarray': [2]}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl -d '{\"data\":{\"ndarray\":[[1.0,2.0,5.0,6.0]]}}' \\\n",
    "         -rpc-header seldon:xgboost -rpc-header namespace:seldon \\\n",
    "         -plaintext \\\n",
    "         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And delete the model we deployed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"xgboost\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/xgboostserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### KFServing V2 protocol\n",
    "\n",
    "We can deploy a XGBoost model, exposing an API compatible with [KFServing's V2 Protocol](https://docs.seldon.io/projects/seldon-core/en/latest/servers/xgboost.html##v2-kfserving-protocol-incubating) by specifying the `protocol` of our `SeldonDeployment` as `kfserving`.\n",
    "For example, we can consider the config below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../testing/resources/iris-xgboost-v2.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../testing/resources/iris-xgboost-v2.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: xgboost\n",
    "spec:\n",
    "  name: iris\n",
    "  protocol: kfserving\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: XGBOOST_SERVER\n",
    "      modelUri: gs://seldon-models/xgboost/iris\n",
    "      name: classifier\n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then apply it to deploy our model to our Kubernetes cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/xgboost created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../testing/resources/iris-xgboost-v2.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"xgboost-default-0-classifier\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"xgboost-default-0-classifier\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=xgboost -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once it's deployed, we can send inference requests to our model.\n",
    "Note that, since it's using the KFServing's V2 Protocol, these requests will be different to the ones using the default Seldon Protocol."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"model_name\": \"classifier\",\n",
      "  \"model_version\": \"v1\",\n",
      "  \"id\": \"7b3fc4b1-bba1-4119-b19a-53649bb7b885\",\n",
      "  \"parameters\": null,\n",
      "  \"outputs\": [\n",
      "    {\n",
      "      \"name\": \"predict\",\n",
      "      \"shape\": [\n",
      "        1\n",
      "      ],\n",
      "      \"datatype\": \"FP32\",\n",
      "      \"parameters\": null,\n",
      "      \"data\": [\n",
      "        2.0\n",
      "      ]\n",
      "    }\n",
      "  ]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "inference_request = {\n",
    "    \"inputs\": [\n",
    "        {\n",
    "          \"name\": \"predict\",\n",
    "          \"shape\": [1, 4],\n",
    "          \"datatype\": \"FP32\",\n",
    "          \"data\": [[1, 2, 3, 4]]\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "endpoint = \"http://localhost:8003/seldon/seldon/xgboost/v2/models/infer\"\n",
    "response = requests.post(endpoint, json=inference_request)\n",
    "\n",
    "print(json.dumps(response.json(), indent=2))\n",
    "assert response.ok"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can delete the model we deployed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"xgboost\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../testing/resources/iris-xgboost-v2.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Tensorflow MNIST Model\n",
    "We can deploy a tensorflow model uploaded to an object store by using the\n",
    "tensorflow model server implementation as the config below.\n",
    "\n",
    "This notebook contains two examples, one which shows how you can use the\n",
    "TFServing prepackaged serve with the Seldon Protocol, and a second one which\n",
    "shows how you can deploy it using the tensorlfow protocol (so you can send\n",
    "requests of the exact format as you would to a tfserving server)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Serve Tensorflow MNIST Model with Seldon Protocol\n",
    "\n",
    "The config file below shows how you can deploy your Tensorflow model which\n",
    "exposes the Seldon protocol."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../servers/tfserving/samples/mnist_rest.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../servers/tfserving/samples/mnist_rest.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: tfserving\n",
    "spec:\n",
    "  name: mnist\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: TENSORFLOW_SERVER\n",
    "      modelUri: gs://seldon-models/tfserving/mnist-model\n",
    "      name: mnist-model\n",
    "      parameters:\n",
    "        - name: signature_name\n",
    "          type: STRING\n",
    "          value: predict_images\n",
    "        - name: model_name\n",
    "          type: STRING\n",
    "          value: mnist-model\n",
    "        - name: model_input\n",
    "          type: STRING\n",
    "          value: images\n",
    "        - name: model_output\n",
    "          type: STRING\n",
    "          value: scores     \n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/tfserving created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/tfserving/samples/mnist_rest.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"tfserving-default-0-mnist-model\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=tfserving -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"tfserving\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "meta {\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 784\n",
      "    values: 0.7363034329885751\n",
      "    values: 0.1214752590440702\n",
      "    values: 0.6811387469595884\n",
      "    values: 0.6532280226091893\n",
      "    values: 0.5409643758278618\n",
      "    values: 0.16215988725534547\n",
      "    values: 0.35479123281330016\n",
      "    values: 0.5607871182857017\n",
      "    values: 0.6010761255325676\n",
      "    values: 0.8902670778749476\n",
      "    values: 0.0868570731397954\n",
      "    values: 0.7904570127547179\n",
      "    values: 0.7122542998518603\n",
      "    values: 0.24630823029939797\n",
      "    values: 0.06771939750935685\n",
      "    values: 0.822122604510554\n",
      "    values: 0.5422718667705746\n",
      "    values: 0.0818978219637192\n",
      "    values: 0.5768929648326077\n",
      "    values: 0.5966960739647604\n",
      "    values: 0.38243947650220445\n",
      "    values: 0.1564417664938903\n",
      "    values: 0.947409102689989\n",
      "    values: 0.5082818863604522\n",
      "    values: 0.8921337711444182\n",
      "    values: 0.9148723260630368\n",
      "    values: 0.6114620528801846\n",
      "    values: 0.743021033845883\n",
      "    values: 0.5153866230874219\n",
      "    values: 0.8936130550396597\n",
      "    values: 0.8211617807532778\n",
      "    values: 0.21219289843893419\n",
      "    values: 0.019821716301115955\n",
      "    values: 0.5043529733203294\n",
      "    values: 0.19167507865391453\n",
      "    values: 0.12475406690801416\n",
      "    values: 0.3741406835139045\n",
      "    values: 0.6261274285447713\n",
      "    values: 0.3401672386072049\n",
      "    values: 0.97152069831883\n",
      "    values: 0.9622678314624786\n",
      "    values: 0.11959892272566952\n",
      "    values: 0.45097548229241957\n",
      "    values: 0.32511650865913433\n",
      "    values: 0.30188998834277314\n",
      "    values: 0.2749736250633016\n",
      "    values: 0.277750645874171\n",
      "    values: 0.5479461817231636\n",
      "    values: 0.3283187525939002\n",
      "    values: 0.5855610972269494\n",
      "    values: 0.26655957605628444\n",
      "    values: 0.17288126000884363\n",
      "    values: 0.8935857109277328\n",
      "    values: 0.13942793456102975\n",
      "    values: 0.8831347671103863\n",
      "    values: 0.23257172060958398\n",
      "    values: 0.9769701669517264\n",
      "    values: 0.5103840128226792\n",
      "    values: 0.3833507985315229\n",
      "    values: 0.7030006728252828\n",
      "    values: 0.6316717523685601\n",
      "    values: 0.30726238758115587\n",
      "    values: 0.20801322126791788\n",
      "    values: 0.05258914613742893\n",
      "    values: 0.5873198085645861\n",
      "    values: 0.4805510872531913\n",
      "    values: 0.09408570817648632\n",
      "    values: 0.7957245092241716\n",
      "    values: 0.24647785740270256\n",
      "    values: 0.7057559036612435\n",
      "    values: 0.3727055847115621\n",
      "    values: 0.00043267798811386093\n",
      "    values: 0.2207011816867367\n",
      "    values: 0.10246738389717558\n",
      "    values: 0.4122518565301895\n",
      "    values: 0.8752870344217444\n",
      "    values: 0.45244305257965123\n",
      "    values: 0.1511521399703143\n",
      "    values: 0.03892680238811774\n",
      "    values: 0.25727701198590147\n",
      "    values: 0.4179870463544978\n",
      "    values: 0.2554594080910736\n",
      "    values: 0.15257408393628857\n",
      "    values: 0.6357521354545026\n",
      "    values: 0.5600895750208378\n",
      "    values: 0.46420853023563347\n",
      "    values: 0.5456799830526327\n",
      "    values: 0.2626279627231338\n",
      "    values: 0.6721570256541571\n",
      "    values: 0.05702250510088902\n",
      "    values: 0.9556804928789183\n",
      "    values: 0.18329720200894828\n",
      "    values: 0.6600997486851473\n",
      "    values: 0.5094924909939877\n",
      "    values: 0.5885032902877644\n",
      "    values: 0.0761636537210646\n",
      "    values: 0.9229153611781235\n",
      "    values: 0.6040705001184103\n",
      "    values: 0.6034264480628279\n",
      "    values: 0.26842117105126273\n",
      "    values: 0.9592349585595005\n",
      "    values: 0.25316858442314905\n",
      "    values: 0.9426193105634458\n",
      "    values: 0.13691086111891693\n",
      "    values: 0.5577352984187683\n",
      "    values: 0.20222115185174583\n",
      "    values: 0.27724039274966994\n",
      "    values: 0.2803057516167521\n",
      "    values: 0.6337285023251188\n",
      "    values: 0.898435524618907\n",
      "    values: 0.3186225805900553\n",
      "    values: 0.6212986217907012\n",
      "    values: 0.4671291715866893\n",
      "    values: 0.5011263298643263\n",
      "    values: 0.16786099387601838\n",
      "    values: 0.6468275445731674\n",
      "    values: 0.9879319070332891\n",
      "    values: 0.5806725008428198\n",
      "    values: 0.6898522651197035\n",
      "    values: 0.8198532932872009\n",
      "    values: 0.7974692145063671\n",
      "    values: 0.005617028749386632\n",
      "    values: 0.9849694249066031\n",
      "    values: 0.4438970664778187\n",
      "    values: 0.42127440627539625\n",
      "    values: 0.10649182916396538\n",
      "    values: 0.5794223897475329\n",
      "    values: 0.386159883321322\n",
      "    values: 0.21831081472531666\n",
      "    values: 0.2976400218499442\n",
      "    values: 0.18252996016955625\n",
      "    values: 0.1310440575389884\n",
      "    values: 0.779049602000965\n",
      "    values: 0.8118435382916147\n",
      "    values: 0.9518506783420652\n",
      "    values: 0.3307219156499994\n",
      "    values: 0.4064675718414933\n",
      "    values: 0.6215987455168476\n",
      "    values: 0.13749082830210668\n",
      "    values: 0.8958081710765494\n",
      "    values: 0.6776843438922747\n",
      "    values: 0.9428933588512107\n",
      "    values: 0.17712126247896365\n",
      "    values: 0.2731590842405416\n",
      "    values: 0.9908055838987785\n",
      "    values: 0.4400399265853929\n",
      "    values: 0.504769964668734\n",
      "    values: 0.02042995949066362\n",
      "    values: 0.6848016902233983\n",
      "    values: 0.015549290873326571\n",
      "    values: 0.0018368307840790354\n",
      "    values: 0.3060353817750894\n",
      "    values: 0.4724422950537789\n",
      "    values: 0.20430971726793623\n",
      "    values: 0.16266423976426658\n",
      "    values: 0.17663836854372217\n",
      "    values: 0.4680444771230422\n",
      "    values: 0.18040874078343816\n",
      "    values: 0.9293332733396308\n",
      "    values: 0.5159037929969271\n",
      "    values: 0.008503973778088869\n",
      "    values: 0.07177944599984187\n",
      "    values: 0.0881504314845869\n",
      "    values: 0.4240221840722792\n",
      "    values: 0.8048319646344456\n",
      "    values: 0.612609623872208\n",
      "    values: 0.5948272849617059\n",
      "    values: 0.2828041478780473\n",
      "    values: 0.4166201897140459\n",
      "    values: 0.585533139631416\n",
      "    values: 0.5488600597322604\n",
      "    values: 0.3210554991018446\n",
      "    values: 0.14429670768723557\n",
      "    values: 0.7453536582567973\n",
      "    values: 0.3635754760171884\n",
      "    values: 0.9096067206579554\n",
      "    values: 0.36371877028467936\n",
      "    values: 0.14986706301051722\n",
      "    values: 0.9420973885805116\n",
      "    values: 0.21759410576226423\n",
      "    values: 0.7547438475151914\n",
      "    values: 0.7974235063133751\n",
      "    values: 0.7158594539826063\n",
      "    values: 0.899232970952357\n",
      "    values: 0.911162338022423\n",
      "    values: 0.554852106062488\n",
      "    values: 0.6610668795884889\n",
      "    values: 0.039105835407815204\n",
      "    values: 0.2887505176217805\n",
      "    values: 0.7294218610072195\n",
      "    values: 0.9482582778754544\n",
      "    values: 0.7798467036813586\n",
      "    values: 0.4946448390124596\n",
      "    values: 0.3073081609286037\n",
      "    values: 0.8868613138790056\n",
      "    values: 0.9223931300714513\n",
      "    values: 0.6420224530266853\n",
      "    values: 0.43420432273336895\n",
      "    values: 0.663471160791883\n",
      "    values: 0.4969977282067015\n",
      "    values: 0.502681883319161\n",
      "    values: 0.1748518334596797\n",
      "    values: 0.8910182583228793\n",
      "    values: 0.5779319908008586\n",
      "    values: 0.7418387386522381\n",
      "    values: 0.8693666629450674\n",
      "    values: 0.0008779918772365836\n",
      "    values: 0.1152722722824926\n",
      "    values: 0.5430881875185213\n",
      "    values: 0.2294847678650619\n",
      "    values: 0.8401091372613628\n",
      "    values: 0.3366053490139074\n",
      "    values: 0.2605818660770459\n",
      "    values: 0.0015441948623255985\n",
      "    values: 0.9522433016737688\n",
      "    values: 0.39162678975408827\n",
      "    values: 0.8029864770397036\n",
      "    values: 0.2745296984611878\n",
      "    values: 0.8599019477588749\n",
      "    values: 0.8746037125915819\n",
      "    values: 0.5829522231941656\n",
      "    values: 0.7694240434242602\n",
      "    values: 0.3530679283018552\n",
      "    values: 0.7386678191422742\n",
      "    values: 0.0026013767293404655\n",
      "    values: 0.9109191806361745\n",
      "    values: 0.3099022115669652\n",
      "    values: 0.9653550969556296\n",
      "    values: 0.8727613493987851\n",
      "    values: 0.8853892496436777\n",
      "    values: 0.1476922535507904\n",
      "    values: 0.32290338569931887\n",
      "    values: 0.8403896028615038\n",
      "    values: 0.9751981122718251\n",
      "    values: 0.8304002435370645\n",
      "    values: 0.8385088151555301\n",
      "    values: 0.1710507878223977\n",
      "    values: 0.369145011183184\n",
      "    values: 0.22098887092105524\n",
      "    values: 0.24605913691421566\n",
      "    values: 0.6954430598889553\n",
      "    values: 0.48989704834011305\n",
      "    values: 0.016752176001942898\n",
      "    values: 0.5392137574536617\n",
      "    values: 0.05722684747074391\n",
      "    values: 0.12486093172148793\n",
      "    values: 0.03628834857321439\n",
      "    values: 0.7660678156478271\n",
      "    values: 0.08921369066237184\n",
      "    values: 0.22861246513095013\n",
      "    values: 0.6558047923379089\n",
      "    values: 0.3450164297492474\n",
      "    values: 0.3773817746004321\n",
      "    values: 0.12075038002883698\n",
      "    values: 0.5549147994051635\n",
      "    values: 0.02197472617312224\n",
      "    values: 0.17860566861412452\n",
      "    values: 0.881628420428201\n",
      "    values: 0.35861987854279176\n",
      "    values: 0.3126728646930961\n",
      "    values: 0.4891052642098741\n",
      "    values: 0.3757873340758542\n",
      "    values: 0.812996090953519\n",
      "    values: 0.8774631419771547\n",
      "    values: 0.0416403869869546\n",
      "    values: 0.2718303533580766\n",
      "    values: 0.6334231753707328\n",
      "    values: 0.35334498487641364\n",
      "    values: 0.48207788113094596\n",
      "    values: 0.0897556140827317\n",
      "    values: 0.9206434150018369\n",
      "    values: 0.7109728501700656\n",
      "    values: 0.9208853632281938\n",
      "    values: 0.06718716468288577\n",
      "    values: 0.27858331777489764\n",
      "    values: 0.0920843006968981\n",
      "    values: 0.5322755519646528\n",
      "    values: 0.6583251445745935\n",
      "    values: 0.6820860938741251\n",
      "    values: 0.24541245641046405\n",
      "    values: 0.7067381757832549\n",
      "    values: 0.15496527139311356\n",
      "    values: 0.5304481812982754\n",
      "    values: 0.7909568078842835\n",
      "    values: 0.45612441548456883\n",
      "    values: 0.3736448929146686\n",
      "    values: 0.8477261822597588\n",
      "    values: 0.5329935305820654\n",
      "    values: 0.47348528893183706\n",
      "    values: 0.07516985453870384\n",
      "    values: 0.05569441693445443\n",
      "    values: 0.020080663675808408\n",
      "    values: 0.5043630565979527\n",
      "    values: 0.8721514813303254\n",
      "    values: 0.1685543049728051\n",
      "    values: 0.3140899996536639\n",
      "    values: 0.1826385914505061\n",
      "    values: 0.27955217586347547\n",
      "    values: 0.2269739318092553\n",
      "    values: 0.30847852964822564\n",
      "    values: 0.15685515997264787\n",
      "    values: 0.09178556081664035\n",
      "    values: 0.3871052061488408\n",
      "    values: 0.8170780136753122\n",
      "    values: 0.6931967816743471\n",
      "    values: 0.26194865347445007\n",
      "    values: 0.4148135155220407\n",
      "    values: 0.3665501570071873\n",
      "    values: 0.9806486967047143\n",
      "    values: 0.9325051629302668\n",
      "    values: 0.4419983566783974\n",
      "    values: 0.09575791923633259\n",
      "    values: 0.003978413629755817\n",
      "    values: 0.8178499377936145\n",
      "    values: 0.4746204569661149\n",
      "    values: 0.17652252428372406\n",
      "    values: 0.44258088399223117\n",
      "    values: 0.7573254838500025\n",
      "    values: 0.7194983658549094\n",
      "    values: 0.2175610763416892\n",
      "    values: 0.2031100593417826\n",
      "    values: 0.43663332513255826\n",
      "    values: 0.06563818854898806\n",
      "    values: 0.7091921330157664\n",
      "    values: 0.5665151493454451\n",
      "    values: 0.6490525785265568\n",
      "    values: 0.27992305357949365\n",
      "    values: 0.48502218851434753\n",
      "    values: 0.7760366546489199\n",
      "    values: 0.6555124038810142\n",
      "    values: 0.8833582629179273\n",
      "    values: 0.8862254304409805\n",
      "    values: 0.4317102871894126\n",
      "    values: 0.2316513753342666\n",
      "    values: 0.09814389854075733\n",
      "    values: 0.5393973993503088\n",
      "    values: 0.5993060561840182\n",
      "    values: 0.4026247146232741\n",
      "    values: 0.3096582221834261\n",
      "    values: 0.6440954731634494\n",
      "    values: 0.015466945586736114\n",
      "    values: 0.4305474307559124\n",
      "    values: 0.3950532150835545\n",
      "    values: 0.22810280568495223\n",
      "    values: 0.9518337688225106\n",
      "    values: 0.5310251766934468\n",
      "    values: 0.8695777842134088\n",
      "    values: 0.3664524385731942\n",
      "    values: 0.5132087217117203\n",
      "    values: 0.07381233253208241\n",
      "    values: 0.6284115968173186\n",
      "    values: 0.7810829725284254\n",
      "    values: 0.5152167875173882\n",
      "    values: 0.6151859486587397\n",
      "    values: 0.16349284423064836\n",
      "    values: 0.5337210082913538\n",
      "    values: 0.852844797208888\n",
      "    values: 0.8939206176097879\n",
      "    values: 0.18408360130741375\n",
      "    values: 0.7881119315939169\n",
      "    values: 0.3772327437517933\n",
      "    values: 0.9921711297411961\n",
      "    values: 0.9292978272888944\n",
      "    values: 0.1823248817044668\n",
      "    values: 0.3308354298685471\n",
      "    values: 0.12766563258133212\n",
      "    values: 0.9906077072824029\n",
      "    values: 0.058522473444330814\n",
      "    values: 0.8942007272598458\n",
      "    values: 0.33542303345300184\n",
      "    values: 0.6405709745315917\n",
      "    values: 0.5377980315334275\n",
      "    values: 0.586493593430412\n",
      "    values: 0.6393065236833644\n",
      "    values: 0.8454767712570775\n",
      "    values: 0.7026664491732846\n",
      "    values: 0.6649462081417578\n",
      "    values: 0.13218821383408563\n",
      "    values: 0.8484346664180982\n",
      "    values: 0.3822819571659164\n",
      "    values: 0.8819388257026116\n",
      "    values: 0.645352582986771\n",
      "    values: 0.8124935265148496\n",
      "    values: 0.590906151350384\n",
      "    values: 0.26116005078750915\n",
      "    values: 0.8131047751072342\n",
      "    values: 0.4480863806431712\n",
      "    values: 0.4221703539794275\n",
      "    values: 0.4356960847017882\n",
      "    values: 0.5445844235359969\n",
      "    values: 0.6162957515075022\n",
      "    values: 0.24814359406506115\n",
      "    values: 0.7345293451499112\n",
      "    values: 0.5539065270799077\n",
      "    values: 0.33804825640762337\n",
      "    values: 0.03458251357057551\n",
      "    values: 0.11431703818615424\n",
      "    values: 0.3679510406563259\n",
      "    values: 0.8184819827720243\n",
      "    values: 0.6558646359109856\n",
      "    values: 0.45318000310358775\n",
      "    values: 0.8981480054708905\n",
      "    values: 0.8461029885125414\n",
      "    values: 0.5069487555806657\n",
      "    values: 0.45978645030305065\n",
      "    values: 0.02352683165516556\n",
      "    values: 0.6018685737820766\n",
      "    values: 0.929110235652851\n",
      "    values: 0.8462760372590702\n",
      "    values: 0.783418309988037\n",
      "    values: 0.9114329754439727\n",
      "    values: 0.5252502879098331\n",
      "    values: 0.4003642029262665\n",
      "    values: 0.365792679698811\n",
      "    values: 0.5430689623573925\n",
      "    values: 0.567011475893137\n",
      "    values: 0.5349356704527894\n",
      "    values: 0.06885927369283784\n",
      "    values: 0.5034041053815209\n",
      "    values: 0.6570480420113727\n",
      "    values: 0.6490980920911288\n",
      "    values: 0.827857684939302\n",
      "    values: 0.5151957621002489\n",
      "    values: 0.5388919042210318\n",
      "    values: 0.09887699966631602\n",
      "    values: 0.9365591129803567\n",
      "    values: 0.7731255838371648\n",
      "    values: 0.5041725739589953\n",
      "    values: 0.9389500437014487\n",
      "    values: 0.2134570824497184\n",
      "    values: 0.9221643236325798\n",
      "    values: 0.41425434297179253\n",
      "    values: 0.7757501838283398\n",
      "    values: 0.7440656005386477\n",
      "    values: 0.4880731418449241\n",
      "    values: 0.09558041709536647\n",
      "    values: 0.33127265816493967\n",
      "    values: 0.32626595124268676\n",
      "    values: 0.8027778677585088\n",
      "    values: 0.28621481396368964\n",
      "    values: 0.7716362202219003\n",
      "    values: 0.6350857195985695\n",
      "    values: 0.963771302509414\n",
      "    values: 0.09505326044072882\n",
      "    values: 0.5581107060550655\n",
      "    values: 0.2777671920459005\n",
      "    values: 0.48234864796805643\n",
      "    values: 0.032694269010350885\n",
      "    values: 0.31045866737248373\n",
      "    values: 0.6674786451763165\n",
      "    values: 0.7969849088272393\n",
      "    values: 0.690142737974502\n",
      "    values: 0.12114949825316823\n",
      "    values: 0.2933836880776354\n",
      "    values: 0.8600304311120042\n",
      "    values: 0.5725897697097084\n",
      "    values: 0.8440501952374988\n",
      "    values: 0.23653402840511972\n",
      "    values: 0.08364681747436253\n",
      "    values: 0.579161767779519\n",
      "    values: 0.01303796967717441\n",
      "    values: 0.9196953357593884\n",
      "    values: 0.21169459975514793\n",
      "    values: 0.19937073697740437\n",
      "    values: 0.03346600814886769\n",
      "    values: 0.11453641266798686\n",
      "    values: 0.7343531498670859\n",
      "    values: 0.016598949031360832\n",
      "    values: 0.997801238265766\n",
      "    values: 0.343158920573423\n",
      "    values: 0.8666756950277428\n",
      "    values: 0.43034419830733595\n",
      "    values: 0.07627643751776847\n",
      "    values: 0.6032811903888042\n",
      "    values: 0.25947676882473325\n",
      "    values: 0.287326154362773\n",
      "    values: 0.2012651954093797\n",
      "    values: 0.9324595668244248\n",
      "    values: 0.5038339871951286\n",
      "    values: 0.3073170280303611\n",
      "    values: 0.9444319556709996\n",
      "    values: 0.6805224683932628\n",
      "    values: 0.7177249897877962\n",
      "    values: 0.6413995182999724\n",
      "    values: 0.5799646157578953\n",
      "    values: 0.17807813893664448\n",
      "    values: 0.3645552237617674\n",
      "    values: 0.10300155812730494\n",
      "    values: 0.5722839966111289\n",
      "    values: 0.9319585549088705\n",
      "    values: 0.23388022271444586\n",
      "    values: 0.0018535755003090681\n",
      "    values: 0.8311310614359066\n",
      "    values: 0.5166449337570462\n",
      "    values: 0.7641018805800096\n",
      "    values: 0.7326293617295236\n",
      "    values: 0.980757108414643\n",
      "    values: 0.725851320640687\n",
      "    values: 0.7366553279783424\n",
      "    values: 0.4925309043473385\n",
      "    values: 0.5728398091998649\n",
      "    values: 0.7866940974691005\n",
      "    values: 0.14847432223625234\n",
      "    values: 0.6269804779497996\n",
      "    values: 0.8149819301994192\n",
      "    values: 0.8503685205366938\n",
      "    values: 0.8757142692001315\n",
      "    values: 0.7788621267247392\n",
      "    values: 0.16324846787271097\n",
      "    values: 0.8466295521163262\n",
      "    values: 0.45918360097903754\n",
      "    values: 0.5420662081019206\n",
      "    values: 0.03123205226078085\n",
      "    values: 0.3280672207128561\n",
      "    values: 0.9879213308527311\n",
      "    values: 0.44632915301986464\n",
      "    values: 0.6396728136928348\n",
      "    values: 0.657241126619934\n",
      "    values: 0.8659182469388128\n",
      "    values: 0.48283024986251355\n",
      "    values: 0.232194282797053\n",
      "    values: 0.37028595494763183\n",
      "    values: 0.3420602200908065\n",
      "    values: 0.366806134877392\n",
      "    values: 0.6986314676641886\n",
      "    values: 0.18227955774159021\n",
      "    values: 0.4658768099878239\n",
      "    values: 0.7618429010040093\n",
      "    values: 0.8521974781638414\n",
      "    values: 0.32970600465280975\n",
      "    values: 0.965641245635317\n",
      "    values: 0.0962729347245429\n",
      "    values: 0.056806259074972054\n",
      "    values: 0.15352297871809284\n",
      "    values: 0.18858115399072584\n",
      "    values: 0.4994306702750998\n",
      "    values: 0.8780039910995361\n",
      "    values: 0.06946508999620526\n",
      "    values: 0.4133408778773222\n",
      "    values: 0.06183940407265598\n",
      "    values: 0.501850212683935\n",
      "    values: 0.911323137342499\n",
      "    values: 0.010340609729104222\n",
      "    values: 0.7002975169450426\n",
      "    values: 0.19229082960854527\n",
      "    values: 0.31157344723131974\n",
      "    values: 0.5672682925797428\n",
      "    values: 0.9563020354938324\n",
      "    values: 0.4865891904612625\n",
      "    values: 0.15610238540536803\n",
      "    values: 0.34189587232437224\n",
      "    values: 0.787585416670521\n",
      "    values: 0.6219562256281694\n",
      "    values: 0.022867724124613575\n",
      "    values: 0.010557368941858547\n",
      "    values: 0.4070462861564792\n",
      "    values: 0.3707136998346672\n",
      "    values: 0.9069467910335517\n",
      "    values: 0.7588476552068967\n",
      "    values: 0.639360016533776\n",
      "    values: 0.5335180404365997\n",
      "    values: 0.5340766181166031\n",
      "    values: 0.45377536728565193\n",
      "    values: 0.32821749665716826\n",
      "    values: 0.7520046618196441\n",
      "    values: 0.5970068195404207\n",
      "    values: 0.5616304334885917\n",
      "    values: 0.9626756636746057\n",
      "    values: 0.3483746521535468\n",
      "    values: 0.753118590653044\n",
      "    values: 0.21131020985396676\n",
      "    values: 0.26709939672746563\n",
      "    values: 0.7255066487694256\n",
      "    values: 0.06280995740625628\n",
      "    values: 0.4310746761801072\n",
      "    values: 0.8346431626372885\n",
      "    values: 0.02910590838586158\n",
      "    values: 0.7742111024189582\n",
      "    values: 0.9671233668527182\n",
      "    values: 0.35638284681998156\n",
      "    values: 0.8196689286166668\n",
      "    values: 0.5952130302556435\n",
      "    values: 0.11587752592451639\n",
      "    values: 0.0980324792202314\n",
      "    values: 0.8480445661278079\n",
      "    values: 0.2659062284884842\n",
      "    values: 0.5508026166257292\n",
      "    values: 0.8426637941076708\n",
      "    values: 0.2739096483997686\n",
      "    values: 0.005846203190377652\n",
      "    values: 0.36268832915400695\n",
      "    values: 0.7732087138470032\n",
      "    values: 0.4370446256365814\n",
      "    values: 0.4065811989750251\n",
      "    values: 0.6382660729057077\n",
      "    values: 0.7729625833341707\n",
      "    values: 0.29594568083151207\n",
      "    values: 0.2920879493272487\n",
      "    values: 0.6040432336090147\n",
      "    values: 0.23828185607789187\n",
      "    values: 0.6772954893913663\n",
      "    values: 0.40735461198936185\n",
      "    values: 0.14400530014017876\n",
      "    values: 0.16270770425364778\n",
      "    values: 0.39705652096380384\n",
      "    values: 0.3578930016084889\n",
      "    values: 0.7199539117585243\n",
      "    values: 0.6413661200291814\n",
      "    values: 0.8351049332812729\n",
      "    values: 0.6823560500331077\n",
      "    values: 0.5270688290375038\n",
      "    values: 0.4026229157256781\n",
      "    values: 0.4248778908953136\n",
      "    values: 0.5761220016410514\n",
      "    values: 0.18358141047612586\n",
      "    values: 0.06708579303685647\n",
      "    values: 0.2101846353815754\n",
      "    values: 0.8469996750833922\n",
      "    values: 0.0052923532744967305\n",
      "    values: 0.835912050845961\n",
      "    values: 0.2955480639821446\n",
      "    values: 0.4415934213481756\n",
      "    values: 0.9218202309743783\n",
      "    values: 0.3065402162851095\n",
      "    values: 0.6662700852768896\n",
      "    values: 0.568669317442409\n",
      "    values: 0.09668581366566886\n",
      "    values: 0.7568371306168005\n",
      "    values: 0.18089149635886392\n",
      "    values: 0.009901837514862954\n",
      "    values: 0.6977585847025538\n",
      "    values: 0.16125520612606548\n",
      "    values: 0.5662402202722648\n",
      "    values: 0.34409219639917443\n",
      "    values: 0.8444930234270572\n",
      "    values: 0.224558252545491\n",
      "    values: 0.4832567996646625\n",
      "    values: 0.934355809188466\n",
      "    values: 0.9682194745085291\n",
      "    values: 0.6977031366623219\n",
      "    values: 0.9731517741227993\n",
      "    values: 0.1926599773562956\n",
      "    values: 0.5270711038565653\n",
      "    values: 0.5227117214032491\n",
      "    values: 0.5062611284099398\n",
      "    values: 0.8139966538856755\n",
      "    values: 0.6412755328551979\n",
      "    values: 0.8778021070307293\n",
      "    values: 0.9549669860572776\n",
      "    values: 0.9921872195423589\n",
      "    values: 0.533312995177466\n",
      "    values: 0.5250237090947721\n",
      "    values: 0.8714735058875508\n",
      "    values: 0.6106783135833788\n",
      "    values: 0.23386881254328862\n",
      "    values: 0.4516218722120472\n",
      "    values: 0.9039266670541178\n",
      "    values: 0.04982603987322565\n",
      "    values: 0.4820616244957956\n",
      "    values: 0.9378882567312806\n",
      "    values: 0.4815414604920272\n",
      "    values: 0.38977931948730704\n",
      "    values: 0.7873297533602746\n",
      "    values: 0.08142964579205336\n",
      "    values: 0.21831765078619636\n",
      "    values: 0.846566989800757\n",
      "    values: 0.2573146933163336\n",
      "    values: 0.15193152763018625\n",
      "    values: 0.44432333804992785\n",
      "    values: 0.8693469109742021\n",
      "    values: 0.4071730817157265\n",
      "    values: 0.5930361954889998\n",
      "    values: 0.22770328934536066\n",
      "    values: 0.3940491646933145\n",
      "    values: 0.44082748540101735\n",
      "    values: 0.4966236586803583\n",
      "    values: 0.3571464752256114\n",
      "    values: 0.320392917230526\n",
      "    values: 0.8053665354022186\n",
      "    values: 0.2775504531400286\n",
      "    values: 0.18682628394609524\n",
      "    values: 0.5102749029987861\n",
      "    values: 0.09758340553375344\n",
      "    values: 0.8261553043948295\n",
      "    values: 0.8899532721131979\n",
      "    values: 0.8215876132245827\n",
      "    values: 0.7741579415995479\n",
      "    values: 0.16912109958027532\n",
      "    values: 0.05287705834056944\n",
      "    values: 0.465283633072741\n",
      "    values: 0.7013167852759242\n",
      "    values: 0.8501986365457948\n",
      "    values: 0.01789170910047644\n",
      "    values: 0.8423338158165913\n",
      "    values: 0.44558139514724704\n",
      "    values: 0.8720108603019613\n",
      "    values: 0.5653905236154808\n",
      "    values: 0.9074132637033316\n",
      "    values: 0.6120958411561497\n",
      "    values: 0.7063238751726747\n",
      "    values: 0.2867387879718889\n",
      "    values: 0.5447958718867185\n",
      "    values: 0.5760076802246712\n",
      "    values: 0.4524351972467908\n",
      "    values: 0.742619582973994\n",
      "    values: 0.8180324754791541\n",
      "    values: 0.5849622568405627\n",
      "    values: 0.38059094000429883\n",
      "    values: 0.09969882389855234\n",
      "    values: 0.5357615945649323\n",
      "    values: 0.9428311446179806\n",
      "    values: 0.6649591018848563\n",
      "    values: 0.6600517321417902\n",
      "    values: 0.14824207373908893\n",
      "    values: 0.5347036588659629\n",
      "    values: 0.48739788231212655\n",
      "    values: 0.5884631079332714\n",
      "    values: 0.13096249347472788\n",
      "    values: 0.5267451259833488\n",
      "    values: 0.19916150794012932\n",
      "    values: 0.8176161847189103\n",
      "    values: 0.9942204801617657\n",
      "    values: 0.1888457156363219\n",
      "    values: 0.2319687636852822\n",
      "    values: 0.7715390568567649\n",
      "    values: 0.565767908957681\n",
      "    values: 0.7689860803394915\n",
      "    values: 0.7191100019020285\n",
      "    values: 0.1554992895883045\n",
      "    values: 0.5355840360566195\n",
      "    values: 0.690487751997688\n",
      "    values: 0.5653295825358345\n",
      "    values: 0.17248202862436723\n",
      "    values: 0.9804170060860974\n",
      "    values: 0.8428880512434194\n",
      "    values: 0.8744115204289816\n",
      "    values: 0.9615067142222875\n",
      "    values: 0.13864809390021027\n",
      "    values: 0.8055159229566986\n",
      "    values: 0.50792978106691\n",
      "    values: 0.1071212246347707\n",
      "    values: 0.38983233891192215\n",
      "    values: 0.04859350406293761\n",
      "    values: 0.6285544269171702\n",
      "    values: 0.8900205041812491\n",
      "    values: 0.920210604508761\n",
      "    values: 0.4839870112848552\n",
      "    values: 0.9971553176417295\n",
      "    values: 0.48733316016452366\n",
      "    values: 0.10361563331333945\n",
      "    values: 0.7091750784132866\n",
      "    values: 0.385821969321797\n",
      "    values: 0.27827176019660793\n",
      "    values: 0.27127272003102276\n",
      "    values: 0.979647801596591\n",
      "    values: 0.8561779932523459\n",
      "    values: 0.29665739688127857\n",
      "    values: 0.5768405913477797\n",
      "    values: 0.7214267898838914\n",
      "    values: 0.833665902514231\n",
      "    values: 0.971137058398432\n",
      "    values: 0.5731901056234175\n",
      "    values: 0.40270025566381573\n",
      "    values: 0.4494944838752235\n",
      "    values: 0.6340960955548102\n",
      "    values: 0.4441342182460881\n",
      "    values: 0.05752189267221164\n",
      "    values: 0.6693195391309364\n",
      "    values: 0.924802859219504\n",
      "    values: 0.24020081122148018\n",
      "    values: 0.19848624349853894\n",
      "    values: 0.21492736829674974\n",
      "    values: 0.5027078154968484\n",
      "    values: 0.8411397256586589\n",
      "    values: 0.3018232619763187\n",
      "    values: 0.03328703868671845\n",
      "    values: 0.4739017508831216\n",
      "    values: 0.4859691263586381\n",
      "    values: 0.6709888247084863\n",
      "    values: 0.10847644786085531\n",
      "    values: 0.6974959086777018\n",
      "    values: 0.41743967971294904\n",
      "    values: 0.9429667287945469\n",
      "    values: 0.2773325749939013\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "{'data': {'names': ['t:0', 't:1', 't:2', 't:3', 't:4', 't:5', 't:6', 't:7', 't:8', 't:9'], 'tensor': {'shape': [1, 10], 'values': [8.24553102e-21, 4.5251533e-33, 0.946396291, 0.0533233285, 7.32232306e-28, 0.000280413544, 5.04385e-15, 1.04924688e-19, 1.38116515e-11, 1.07046032e-18]}}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,784))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1, 784], 'values': [0.13419748086376349, 0.23178703054633498, 0.44223572424534485, 0.2696273680588255, 0.07152377975849233, 0.5727816756904325, 0.39309147982375336, 0.3721411402484911, 0.551244898235985, 0.43975687243817896, 0.8415874250304912, 0.01724839861848615, 0.9992444311222564, 0.20116248615583598, 0.8134491993637052, 0.7874830465957086, 0.6030254329128775, 0.7379539664731155, 0.9732382705725281, 0.3376471422948141, 0.4450901915855564, 0.4104062891617003, 0.6235508101202012, 0.5280596447975435, 0.30707052711675076, 0.8616168522237181, 0.08748039835534571, 0.6441019299651144, 0.2090627479548135, 0.22655911024713882, 0.26494800039445654, 0.0014838612813646845, 0.257641691814799, 0.31542040987433173, 0.9071529110894798, 0.1368715432376224, 0.30286691205277494, 0.3732340970325615, 0.7286908117093446, 0.013476068707320321, 0.37852914951760863, 0.6529607177918756, 0.46718776396978656, 0.510428270339585, 0.8885521600847648, 0.9631657554723222, 0.8397531156094977, 0.2400589726502229, 0.15217775804062306, 0.46429846926154905, 0.08385139261432639, 0.7307691441888393, 0.589316956924941, 0.6730002455166538, 0.23707054406597605, 0.2931998272857852, 0.1100261639851291, 0.4571500381453225, 0.9090837879625789, 0.7003948703248867, 0.4048636706049221, 0.20974312769276438, 0.7904716141162355, 0.09386522890806215, 0.6311302357089936, 0.11797146474945008, 0.4260216271737973, 0.05903376947431582, 0.577354102399464, 0.29522656968064576, 0.1304185252384179, 0.0402417436484418, 0.03864471534126657, 0.14417614489918817, 0.8476837959336987, 0.9833408642735735, 0.6437879802401147, 0.9142763638604318, 0.1651651384063122, 0.5804354228381654, 0.08298035917450908, 0.21136462217640872, 0.828392469670638, 0.25912168273289016, 0.0916760221909485, 0.8596989257230342, 0.32305668061918646, 0.4046074835135497, 0.9608387223649623, 0.183638735095903, 0.26065807137086616, 0.5255957209136339, 0.32977839963704225, 0.673435476820454, 0.29048277105575837, 0.29361495330687126, 0.21521087630301783, 0.363162392591207, 0.10027524213249717, 0.2810064552832071, 0.16255720758870518, 0.9150414758763193, 0.5467904004242236, 0.3376724740024136, 0.8359835446304251, 0.864067022501903, 0.1411051067850705, 0.49763389434893457, 0.057020469536536944, 0.17522294212046863, 0.7379504970932488, 0.11710269292509523, 0.7568065252048743, 0.8131416284960465, 0.2847692741078598, 0.9287977416873879, 0.2690057105685779, 0.6630207192864265, 0.7951751447879697, 0.4926187678769478, 0.9986232840119157, 0.0527749975505315, 0.7363266702968961, 0.047407527893227996, 0.9290682552601192, 0.3094042639267317, 0.7241233924568273, 0.03217104315809516, 0.06277402102324225, 0.9435428045839565, 0.38994336229894755, 0.63561786633812, 0.46852277961980804, 0.5736339512054862, 0.8439305231547688, 0.757517562341326, 0.837971682226475, 0.6905635299878967, 0.13821404470992105, 0.8888564536075351, 0.12974282880408627, 0.05442343901051461, 0.7427887067230462, 0.9375927257367739, 0.40500673472929116, 0.6023211168608983, 0.8962966762499802, 0.5731531894040041, 0.01216967970721805, 0.25389988530699203, 0.588530242997082, 0.6444057552000971, 0.5902255859359075, 0.7110459694277497, 0.291112819113845, 0.9863630862680458, 0.13949477587965275, 0.9350209775543827, 0.6909252374668218, 0.7768292256296734, 0.030574950082405183, 0.9394454028933794, 0.827330718093766, 0.8780177829268756, 0.6073590887616083, 0.4446358091522632, 0.4439942381012457, 0.04473784809297621, 0.36366613206312837, 0.99297143694405, 0.9470692093745043, 0.799000981931117, 0.5248895935812256, 0.568400901305973, 0.14234450112994212, 0.5936690687022849, 0.24911083166347348, 0.46352114120563936, 0.09011556513018981, 0.0604828524992268, 0.1581682310207142, 0.6977007639177987, 0.5100596435446672, 0.4291847340392112, 0.6276153278270705, 0.8743440237097942, 0.0925281957246884, 0.9378787267867864, 0.7947455887038958, 0.07773080435722601, 0.8298692196173032, 0.7776104854687146, 0.5676922413328264, 0.721837030974071, 0.4260377260409577, 0.27828636370783866, 0.9494614586141553, 0.5719029244522904, 0.5062535345458372, 0.685169436403047, 0.4011270478100226, 0.6864621367404945, 0.8701642292022752, 0.9335405488361644, 0.350404545918959, 0.3095478416202325, 0.04708604181674436, 0.9782911017776409, 0.5092156419430964, 0.17345993482101962, 0.7702577835361663, 0.07391146561068773, 0.37314128311492756, 0.5630724682602296, 0.6611900137192385, 0.4550815593049843, 0.24996757329671737, 0.7490758105122737, 0.4764235396829529, 0.9068398446804367, 0.30606896504773273, 0.4933195806185472, 0.8720788126694032, 0.2200571019512897, 0.3332171034506459, 0.8938275646200261, 0.3941318725816987, 0.9305428645461274, 0.8547422151309964, 0.7610948054802418, 0.5450274559363226, 0.7812321495574984, 0.721591718123395, 0.3118697798159361, 0.6389366719736986, 0.2455052862350423, 0.5141467582986424, 0.5140383586563555, 0.7950439162935117, 0.8322605859238638, 0.9460047298648459, 0.6496618782840512, 0.7250928919342197, 0.600711666218656, 0.36589447919908547, 0.5013115138931293, 0.8423952810038424, 0.7495944138702135, 0.5414015631811495, 0.8859161323897734, 0.5965360566207157, 0.40939191689630294, 0.8258146869751216, 0.4758439481221255, 0.6850382806657973, 0.28262025638937427, 0.8291672928132477, 0.9514854975904402, 0.3570960791216594, 0.5240280110984059, 0.2548171273514158, 0.005514776946487165, 0.2869859061402711, 0.3426917289072575, 0.8385076894328787, 0.2456088043416782, 0.7718858314533144, 0.6807926232979943, 0.6074714331647073, 0.9320766322638997, 0.1981004600417372, 0.9685290705959322, 0.6276023219618522, 0.7928177443397391, 0.5812462238274193, 0.9661887233138354, 0.35500371654709717, 0.6851925314965512, 0.15291726695147578, 0.5643314277821121, 0.915860799871432, 0.6681819667855035, 0.9235130137328863, 0.6162799375968487, 0.2053218212168053, 0.747627254018538, 0.7741238895120719, 0.7614039187089631, 0.34096955671454976, 0.08305687288364294, 0.6157027439423446, 0.7693197888442768, 0.27101003675252633, 0.8535235660082907, 0.68545081604543, 0.9245618195747557, 0.9936763942293515, 0.7268949686596877, 0.26304911438715484, 0.2582632715723163, 0.8245133983152992, 0.4969574045835582, 0.07076870235459198, 0.1717019358681705, 0.9103713125056555, 0.035548415587356996, 0.8022922138678616, 0.2623264629155614, 0.935277135482782, 0.1555657182301483, 0.2740573068399339, 0.7724396549061213, 0.3795536849636334, 0.3792126611223582, 0.42791425521380544, 0.26375410181640757, 0.30450066981132706, 0.12938307472263977, 0.971088275052168, 0.6395715315575684, 0.9139275561914801, 0.44766105265551426, 0.8532873595647018, 0.4952756741265133, 0.5392716067338299, 0.46814276211774974, 0.9465706797973259, 0.20520169475857342, 0.28594102114633557, 0.9268549727235904, 0.24338518151765687, 0.7697234895474201, 0.051657957432591495, 0.6685610815342509, 0.712627313905724, 0.5717113415910094, 0.9743320950212041, 0.8433921210071532, 0.2644246965878597, 0.5958313067481154, 0.8209945400445483, 0.8756115249432307, 0.5545926762548329, 0.9636504197860154, 0.99218090609753, 0.32153465217338684, 0.16485025672238462, 0.1876123839029854, 0.05637539290599536, 0.6418480444166167, 0.19609964343994857, 0.29915828766386077, 0.00544916422687991, 0.3272291754149018, 0.3363747917737613, 0.6702214382869357, 0.49135242108126154, 0.9403692088218855, 0.35093862353895977, 0.7769156916514725, 0.6831233035217892, 0.9913218919859279, 0.48483738575757784, 0.6563741163920626, 0.2732086999848822, 0.6390184970854177, 0.924729429725906, 0.12020001852526185, 0.8644102489208366, 0.618419628601709, 0.2657150862601382, 0.0645555261004277, 0.9873724671625413, 0.8597728509599905, 0.11942629770364188, 0.8936693406167003, 0.16901804395428544, 0.40679419992032606, 0.2571308249813359, 0.6769648108246318, 0.9389040131993519, 0.8092050397985746, 0.45035699913661353, 0.7898695508347594, 0.5748908675655702, 0.4788095961138882, 0.5375032269521436, 0.1381960188485155, 0.7036280305083004, 0.5095084848570596, 0.5777982893682614, 0.7085356033785147, 0.48648228401838667, 0.7047670539427573, 0.7743581117232212, 0.9679690415189439, 0.08151244733684115, 0.14927180607395285, 0.8364569393485999, 0.03586424015775602, 0.0375648487402348, 0.16208766290448062, 0.812864896568399, 0.5765879863535578, 0.4145064357963444, 0.9786524337098816, 0.7812542981552282, 0.6984721265490218, 0.19223394826169937, 0.12705179016712675, 0.7047133431081949, 0.28924816934194597, 0.01438027789261509, 0.9760840189458634, 0.9456115101949265, 0.008008235544225362, 0.6196172657698679, 0.752606063300519, 0.3042893646310051, 0.47035283749915335, 0.349157131342577, 0.9187353199499827, 0.715301996632876, 0.6711573634914451, 0.45724277613979936, 0.29852642255130846, 0.13071598945162421, 0.8246095151232121, 0.19241023151323622, 0.237048570834967, 0.5714968995009478, 0.7198890340116956, 0.4608449064948362, 0.48511145558165203, 0.07737540836364387, 0.28830377684757735, 0.8862109400328939, 0.4118775271150862, 0.7851702052630384, 0.4978626169445246, 0.3501166597961566, 0.7278215207089693, 0.7961136975652259, 0.04793584788967442, 0.7811210274825605, 0.7936016942093618, 0.5025980858945263, 0.8118361735157303, 0.6072076898907884, 0.2872703012235671, 0.7743659234596992, 0.9666854079042979, 0.9707955303131107, 0.10164765831717748, 0.2179756883294286, 0.6323142437241294, 0.9250477777292629, 0.860592365797268, 0.9137132750719308, 0.914427657740456, 0.42481577683894534, 0.22849200873182296, 0.08203829858993128, 0.054142483010120035, 0.34318155845804144, 0.8996678773447693, 0.8896875100316288, 0.2311211639737233, 0.39188975370869505, 0.42451279532085684, 0.6873428310679885, 0.01411452557855064, 0.6066191274906801, 0.1843595887551357, 0.9202126114972693, 0.2792248820915547, 0.7198494569034175, 0.4239943262876391, 0.580595181484383, 0.9394277165027181, 0.24866731627085836, 0.5374706485150308, 0.09188566912822804, 0.8466456912004529, 0.9068463130275841, 0.3381084509482324, 0.8706903087147493, 0.010003767154691756, 0.08236726601018407, 0.3306220249669857, 0.3367853288119069, 0.245933324997466, 0.628400190691775, 0.6882132730025499, 0.07876749241357783, 0.5802464797224822, 0.1280882270591519, 0.9310804657237919, 0.16691536012743868, 0.19373586015022448, 0.6837345891906627, 0.6485681595748786, 0.6462410862517849, 0.6383252352477646, 0.816198668375224, 0.16341884559678554, 0.9280661105058036, 0.5760325488698934, 0.7993285704199959, 0.364146114494698, 0.35310667902638415, 0.49416741992432034, 0.5720130728750026, 0.23830973594914917, 0.8061749925053503, 0.6997839858727128, 0.39946989764653273, 0.8843232854997451, 0.9835067055774989, 0.9194941539551212, 0.6154033940167327, 0.9358393295077562, 0.15901372520700363, 0.0064982684627012954, 0.039949033408459456, 0.5229006056935335, 0.33854113630587657, 0.4530944521402853, 0.5892986018785812, 0.285144089065137, 0.11113485108719545, 0.07852994139925085, 0.3194885138416229, 0.9950244407635428, 0.7253671820365413, 0.8285344371186119, 0.4217372276776403, 0.34549824281382835, 0.96582359317704, 0.5239546952097599, 0.17047437108571706, 0.6207719849919786, 0.8437347354985423, 0.9064511363063429, 0.5406786618004253, 0.3089643675239484, 0.952785344567674, 0.48918049508928263, 0.1001783164686415, 0.08337337150294644, 0.8070520026841039, 0.11862291185411444, 0.6319550109774433, 0.018552139796252476, 0.35321812308963796, 0.6338678353755586, 0.6023659185059351, 0.3924108118061381, 0.4717090668632625, 0.8529291829296302, 0.016215949312506805, 0.8566205550538725, 0.5255306894801209, 0.4260179921210584, 0.17558177079227277, 0.994334247821781, 0.3456401869579079, 0.981609867378798, 0.6843569891126943, 0.3014564829683495, 0.7852904204575816, 0.047573609698187136, 0.7869659347379302, 0.8847998147957128, 0.5380690917351099, 0.21631745289915938, 0.641772598200905, 0.9628355322741329, 0.8029920786138016, 0.01963584792924522, 0.18029852707226768, 0.4006729542839288, 0.8696413645042306, 0.2976194097977076, 0.5206014903496996, 0.5391398200820117, 0.21899399054640578, 0.7032856883376074, 0.8138691373919892, 0.3569605698587883, 0.8046517574089153, 0.05634902198905278, 0.9904507942743307, 0.05669283533200087, 0.18326041826130024, 0.7217152129836034, 0.9054913391319004, 0.3120645462065067, 0.33465933361059685, 0.8073765259810278, 0.5355056273175401, 0.3387554049068594, 0.21380498605960663, 0.9217442083047366, 0.45616289284801137, 0.7186403801721876, 0.08308915973410169, 0.7144762814346852, 0.4764522126857411, 0.21423289165017445, 0.6628411687825407, 0.8989989669388376, 0.8685543778594821, 0.99014398778506, 0.6566533931583192, 0.06679080101344481, 0.43711199946870927, 0.7213849294856659, 0.6881603165814358, 0.19192921366503135, 0.6041799453183039, 0.6786655730911394, 0.5925109143417246, 0.8286886996106787, 0.7459531095875908, 0.9676860303005279, 0.6070038353167432, 0.18749272597298017, 0.023467210043108144, 0.9429368241512177, 0.3683256597099517, 0.8418884226971971, 0.37563635146055874, 0.08346457383057104, 0.7924049500341593, 0.9523218815003489, 0.42056257000936714, 0.31710010593166316, 0.5209291849554504, 0.3054543813109226, 0.8529895086050358, 0.015859825243945136, 0.0695085829808858, 0.5964312775644822, 0.3096080538503396, 0.4170594801813742, 0.9692727953514577, 0.04745144770109566, 0.16835604895478196, 0.14826184180440516, 0.18139246159292655, 0.2897308291060634, 0.9710562363915802, 0.20596676818695292, 0.5616341871260482, 0.2967589027797687, 0.14741486533035442, 0.5638937040823886, 0.1572038040192989, 0.8339180375720128, 0.46549303242703854, 0.33297890864645585, 0.6109799780818991, 0.9463949719810457, 0.8373716435347347, 0.43336314240698826, 0.326205549994909, 0.7492867448161169, 0.5805528201986576, 0.9135760352096932, 0.8882686979641876, 0.778412230983423, 0.8423712387393348, 0.786434713957904, 0.6488384586720363, 0.01709790925582344, 0.06131963846139532, 0.9906335966508523, 0.750411195868309, 0.3039133812534127, 0.20871207962826976, 0.731105829386349, 0.03218453216768358, 0.7820732391262731, 0.36800946547789337, 0.5760777600607387, 0.9616550291513201, 0.34460839649633745, 0.9377034943265039, 0.5996277805793678, 0.7520155306590419, 0.6526882963442419, 0.009603685475125023, 0.3576186537305278, 0.7796664423765489, 0.1370629144006562, 0.1013555207644532, 0.1968634715746843, 0.5288716744583375, 0.14413108857816748, 0.5961924154002782, 0.46093711094963896, 0.604070675204651, 0.6274233546195345, 0.9100636321723097, 0.6727952350597669, 0.20209560518176173, 0.09012897708783729, 0.356297761146382, 0.8306667050929043, 0.7310803002441775, 0.9699845907795751, 0.9848909580550989, 0.40130179350144257, 0.011880317907041138, 0.7887265099873977, 0.34817797317297494, 0.48845362702571327, 0.0640409025326999, 0.9856655801004994, 0.3302581397189761, 0.8598339677374782, 0.6255506264025438, 0.42190914816029745, 0.3128490582092297, 0.6780369388284762, 0.23736299186645649, 0.3298380933933398, 0.7169207698287051, 0.9105842831539935, 0.5485800911415574, 0.3818045779962158, 0.3619240207612451, 0.27976669397345466, 0.43409198789818415, 0.26507409062466136, 0.271526267842626, 0.8649061173187783, 0.27457260930600813, 0.33548331587138824, 0.7824992983794352, 0.7407967828917424, 0.7582145111668295, 0.4131362768672391, 0.32774785288208474, 0.610201288427749, 0.6630946925388368, 0.8818399352001711, 0.7597753793404299, 0.7590560870750487, 0.592893684049848, 0.1756237444339177, 0.7244459316989478, 0.8837821321022633, 0.2076159923038886, 0.42134751818127636, 0.026085101747561512, 0.048731112776937646, 0.669949986614359, 0.9306260960496128, 0.3988904516078361, 0.5313011506080507, 0.012283310001061198, 0.7562697524384102, 0.6190655417029755, 0.5730963683170917, 0.2805193405792371, 0.48093439954426076, 0.536840334945942, 0.9522378284441914, 0.37681736932432985, 0.3525534485422056, 0.8825456040537141, 0.7343081942071908, 0.6132885981222034, 0.9733495689368262, 0.47066705175281964, 0.19986437943761404, 0.42840262346906377, 0.7689191797361147, 0.833016638662058, 0.34280087961464145, 0.1567313948484702, 0.1264898532290455]}}}\n",
      "Response:\n",
      "{'data': {'tftensor': {'dtype': 'DT_FLOAT', 'tensorShape': {'dim': [{'size': '1'}, {'size': '10'}]}, 'floatVal': [1.1139304e-18, 7.1242203e-34, 0.18723784, 0.0022280787, 3.8356077e-29, 0.8105242, 3.0204097e-14, 1.5520727e-16, 9.876945e-06, 1.8018426e-18]}}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"grpc\",shape=(1,784))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And delete the model we deployed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"tfserving\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/tfserving/samples/mnist_rest.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Serve Tensorflow Model with Tensorflow protocol\n",
    "\n",
    "The config file below shows how you can deploy your Tensorflow model which\n",
    "exposes the Tensorflow protocol."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../servers/tfserving/samples/halfplustwo_rest.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../servers/tfserving/samples/halfplustwo_rest.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: hpt\n",
    "spec:\n",
    "  name: hpt\n",
    "  protocol: tensorflow\n",
    "  transport: rest\n",
    "  predictors:\n",
    "  - graph:\n",
    "      children: []\n",
    "      implementation: TENSORFLOW_SERVER\n",
    "      modelUri: gs://seldon-models/tfserving/half_plus_two\n",
    "      name:  halfplustwo\n",
    "      parameters:\n",
    "        - name: model_name\n",
    "          type: STRING\n",
    "          value: halfplustwo\n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/hpt created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/tfserving/samples/halfplustwo_rest.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"hpt-default-0-halfplustwo\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=hpt -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'predictions': [2.5, 3.0, 4.5]}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "X=!curl -s -d '{\"instances\": [1.0, 2.0, 5.0]}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/hpt/v1/models/halfplustwo/:predict \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)\n",
    "assert(d[\"predictions\"][0] == 2.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'outputs': {'x': {'dtype': 'DT_FLOAT', 'tensorShape': {'dim': [{'size': '3'}]}, 'floatVal': [2.5, 3, 3.5]}}, 'modelSpec': {'name': 'halfplustwo', 'version': '123', 'signatureName': 'serving_default'}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl \\\n",
    "   -d '{\"model_spec\":{\"name\":\"halfplustwo\"},\"inputs\":{\"x\":{\"dtype\": 1, \"tensor_shape\": {\"dim\":[{\"size\": 3}]}, \"floatVal\" : [1.0, 2.0, 3.0]}}}' \\\n",
    "   -rpc-header seldon:hpt -rpc-header namespace:seldon \\\n",
    "   -plaintext -proto ./prediction_service.proto \\\n",
    "   0.0.0.0:8003 tensorflow.serving.PredictionService/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)\n",
    "assert(d[\"outputs\"][\"x\"][\"floatVal\"][0] == 2.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"hpt\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/tfserving/samples/halfplustwo_rest.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve MLFlow Elasticnet Wines Model\n",
    "We can deploy an MLFlow model uploaded to an object store by using the MLFlow\n",
    "model server implementation as the config below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting ../servers/mlflowserver/samples/elasticnet_wine.yaml\n"
     ]
    }
   ],
   "source": [
    "%%writefile ../servers/mlflowserver/samples/elasticnet_wine.yaml\n",
    "apiVersion: machinelearning.seldon.io/v1alpha2\n",
    "kind: SeldonDeployment\n",
    "metadata:\n",
    "  name: mlflow\n",
    "spec:\n",
    "  name: wines\n",
    "  predictors:\n",
    "  - componentSpecs:\n",
    "    - spec:\n",
    "        # We are setting high failureThreshold as installing conda dependencies\n",
    "        # can take long time and we want to avoid k8s killing the container prematurely\n",
    "        containers:\n",
    "        - name: classifier\n",
    "          livenessProbe:\n",
    "            initialDelaySeconds: 80\n",
    "            failureThreshold: 200\n",
    "            periodSeconds: 5\n",
    "            successThreshold: 1\n",
    "            httpGet:\n",
    "              path: /health/ping\n",
    "              port: http\n",
    "              scheme: HTTP\n",
    "          readinessProbe:\n",
    "            initialDelaySeconds: 80\n",
    "            failureThreshold: 200\n",
    "            periodSeconds: 5\n",
    "            successThreshold: 1\n",
    "            httpGet:\n",
    "              path: /health/ping\n",
    "              port: http\n",
    "              scheme: HTTP\n",
    "    graph:\n",
    "      children: []\n",
    "      implementation: MLFLOW_SERVER\n",
    "      modelUri: gs://seldon-models/mlflow/elasticnet_wine\n",
    "      name: classifier\n",
    "    name: default\n",
    "    replicas: 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/mlflow created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/mlflowserver/samples/elasticnet_wine.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"mlflow-default-0-classifier\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=mlflow -o jsonpath='{.items[0].metadata.name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### REST requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'data': {'names': [], 'ndarray': [5.275558760255382]}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "X=!curl -s -d '{\"data\": {\"ndarray\":[[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,1.1]]}}' \\\n",
    "   -X POST http://localhost:8003/seldon/seldon/mlflow/api/v1.0/predictions \\\n",
    "   -H \"Content-Type: application/json\"\n",
    "d=json.loads(X[0])\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"mlflow\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "meta {\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 11\n",
      "    values: 0.2221219485862146\n",
      "    values: 0.48347468791705017\n",
      "    values: 0.792909901124416\n",
      "    values: 0.8575020679326859\n",
      "    values: 0.3393108788709739\n",
      "    values: 0.9524947980687155\n",
      "    values: 0.4764630378302829\n",
      "    values: 0.3330827286255519\n",
      "    values: 0.02818894894121715\n",
      "    values: 0.20379458867010114\n",
      "    values: 0.5176887266713507\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "{'data': {'names': [], 'tensor': {'shape': [1], 'values': [5.223443971007788]}}, 'meta': {}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,11))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### gRPC Requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {}, 'data': {'ndarray': [5.275558760255382]}}\n"
     ]
    }
   ],
   "source": [
    "X=!cd ../executor/proto && grpcurl -d '{\"data\":{\"ndarray\":[[0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,1.1]]}}' \\\n",
    "         -rpc-header seldon:mlflow -rpc-header namespace:seldon \\\n",
    "         -plaintext \\\n",
    "         -proto ./prediction.proto  0.0.0.0:8003 seldon.protos.Seldon/Predict\n",
    "d=json.loads(\"\".join(X))\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1, 11], 'values': [0.3862249013971256, 0.41803382328603056, 0.5006540243492418, 0.22082451828445948, 0.25340037683662653, 0.23224919838688673, 0.23156224762583522, 0.7652379449586778, 0.590985787494962, 0.7813227013394188, 0.48042753535482696]}}}\n",
      "Response:\n",
      "{'meta': {}, 'data': {'tensor': {'shape': [1], 'values': [5.214988433508114]}}}\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"grpc\",shape=(1,11))\n",
    "print(r)\n",
    "assert(r.success==True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"mlflow\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/mlflowserver/samples/elasticnet_wine.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}