fix v2 serving readme and add example (#481)

Co-authored-by: yaron haviv <yaronh@iguazio.com>

examples/v2_model_server.ipynb

@@ -0,0 +1,350 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# V2 Model Server (SKLearn)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "> 2020-10-12 14:52:57,535 [warning] Failed resolving version info. Ignoring and using defaults\n"
+     ]
+    }
+   ],
+   "source": [
+    "import mlrun"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Configuration and package dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "%nuclio: setting kind to 'serving'\n",
+      "%nuclio: setting spec.build.baseImage to 'mlrun/mlrun'\n"
+     ]
+    }
+   ],
+   "source": [
+    "%nuclio config kind=\"serving\"\n",
+    "%nuclio config spec.build.baseImage = \"mlrun/mlrun\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%nuclio cmd -c\n",
+    "python -m pip install numpy cloudpickle v3io sklearn"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Serving class code"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from cloudpickle import load\n",
+    "import numpy as np\n",
+    "from typing import List"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class ClassifierModel(mlrun.serving.V2ModelServer):\n",
+    "    def load(self):\n",
+    "        \"\"\"load and initialize the model and/or other elements\"\"\"\n",
+    "        model_file, extra_data = self.get_model('.pkl')\n",
+    "        self.model = load(open(model_file, 'rb'))\n",
+    "\n",
+    "    def predict(self, body: dict) -> List:\n",
+    "        \"\"\"Generate model predictions from sample.\"\"\"\n",
+    "        feats = np.asarray(body['inputs'])\n",
+    "        result: np.ndarray = self.model.predict(feats)\n",
+    "        return result.tolist()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# nuclio: end-code"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Convert to function object"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The sklearn-project generated one or more models that will be deployed in the server project `sklearn-servers`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "models_path = 'https://s3.wasabisys.com/iguazio/models/iris/model.pkl'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Document and save (as a template)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "> 2020-10-12 14:47:46,331 [info] function spec saved to path: function.yaml\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<mlrun.runtimes.serving.ServingRuntime at 0x7f9da3564090>"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "fn = mlrun.code_to_function('v2-model-server', description=\"generic sklearn model server\",\n",
+    "                     categories=['serving', 'ml'],\n",
+    "                     labels={'author': 'yaronh', 'framework': 'sklearn'},\n",
+    "                     code_output='.')\n",
+    "fn.spec.default_class = 'ClassifierModel'\n",
+    "#print(fn.to_yaml())\n",
+    "fn.export()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Configure and add model(s)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "fn.add_model('mymodel', model_path=models_path)\n",
+    "#fn.verbose = True"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Test models locally (using a server emulator)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# create an emulator (mock server) from the function configuration)\n",
+    "server = fn.to_mock_server(globals())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Test against the iris dataset "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sklearn.datasets import load_iris\n",
+    "iris = load_iris()\n",
+    "x = iris['data'].tolist()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "> 2020-10-12 14:44:19,297 [debug] router run model mymodel, op=infer\n",
+      "> 2020-10-12 14:44:19,297 [debug] router run model mymodel, op=infer\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "dict_keys(['id', 'model_name', 'outputs'])"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "result = server.test(\"/v2/models/mymodel/infer\", {\"inputs\": x})\n",
+    "result.keys()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "##  Deploy server"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "> 2020-10-12 14:44:20,662 [info] deploy started\n",
+      "[nuclio] 2020-10-12 14:44:21,772 (info) Build complete\n",
+      "[nuclio] 2020-10-12 14:44:29,852 (info) Function deploy complete\n",
+      "[nuclio] 2020-10-12 14:44:29,859 done updating v2-srv-v2-model-server, function address: 3.128.234.166:30830\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'http://3.128.234.166:30830'"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "fn.apply(mlrun.mount_v3io())\n",
+    "fn.deploy(project='v2-srv')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "##  Test server"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'id': 'bd688fa9-38cc-4a5d-95e8-1445bdd1520a',\n",
+       " 'model_name': 'mymodel',\n",
+       " 'outputs': [0, 2]}"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "my_data = '''{\"inputs\":[[5.1, 3.5, 1.4, 0.2],[7.7, 3.8, 6.7, 2.2]]}'''\n",
+    "fn.invoke('/v2/models/mymodel/infer', my_data)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

mlrun/serving/README.md

@@ -23,7 +23,7 @@ but eliminate the operational overhead and provide additional functionality.
 - [Model Server API](#model-server-api)
 - [Model Monitoring](#model-monitoring)
 
-<b>You can find examples for serving functions ..TBD..</b>
+<b>You can find [this notebook example](../../examples/v2_model_server.ipynb) with a V2 serving functions</b>
 
 ## Creating And Serving A Model
 
@@ -109,6 +109,10 @@ x = load_iris()['data'].tolist()
 result = server.test("mymodel/infer", {"inputs": x})
 ```
 
+> Note: you can also create `mock_server` object from the function object 
+using `fn.to_mock_server()`, this way the mock_server configuration will 
+be identical to the function spec.
+
 #### Load() method
 
 in the load method we download the model from external store, run the algorithm/framework
@@ -162,7 +166,7 @@ and aggregate the result), multi-armed-bandit, etc.
 You can use a pre-defined Router class, or write your own custom router. 
 Router can route to models on the same function or access models on a separate function.
 
-to specify the router class and class args use `.set_router()` with your function.
+to specify the topology, router class and class args use `.set_topology()` with your function.
 
 ## Creating Model Serving Function (Service)
 
@@ -199,6 +203,13 @@ to deploy a model we can simply call:
 fn.deploy()
 ```
 
+to create a `mock server` for testing from the function spec use `fn.to_mock_server()`, example:
+
+```python
+server = fn.to_mock_server(globals())
+result = server.test("/v2/models/mymodel/infer", {"inputs": x})
+```
+
 we can also deploy a model from within an ML pipeline (check the various demos for details).
 
 ## Model Server API