Add example helm charts for inference graphs

docs/getting_started/readme.md

@@ -21,16 +21,20 @@ To allow users to easily wrap machine learning components built using different
 
 # Define Runtime Service Graph
 
-To run your machine learning graph on Kubernetes you need to define how the components you created in the last step fit together to represent a service graph. This is defined inside a [SeldonDeployment Kubernetes Custom resource](../reference/seldon-deployment.md). A [guide to constructing this custom resource service graph is provided](../crd/readme.md).
+To run your machine learning graph on Kubernetes you need to define how the components you created in the last step fit together to represent a service graph. This is defined inside a [SeldonDeployment Kubernetes Custom resource](../reference/seldon-deployment.md). A [guide to constructing this custom resource service graph is provided](../inference-graph.md).
 
 ![graph](./graph.png)
 
 # Deploy and Serve Predictions
 
 You can use ```kubectl``` to deploy your ML service like any other Kubernetes resource. This is discussed [here](../deploying.md).
 
-# Worked Examples
+# Next Steps
 
  * [Jupyter notebooks showing worked examples](../../readme.md#quick-start)
+ * Templated Helm Charts:
+   * [Single Model with optional Outlier Detector](../../helm-charts/seldon-single-model/README.md)
+   * [AB Test between two models](../../helm-charts/seldon-abtest/README.md)
+   * [Multi-Armed Bandit of two models](../../helm-charts/seldon-mab/README.md)   
  * [Integration with other machine learning frameworks](../../readme.md#integrations)
 

docs/inference-graph.md

@@ -0,0 +1,49 @@
+# Inference Graph
+
+Seldon Core extends Kubernetes with its own custom resource SeldonDeployment where you can define your runtime inference graph made up of models and other components that Seldon will manage.
+
+A SeldonDeployment is a JSON or YAML file that allows you to define your graph of component images and the resourcs each of those images will need to run (using a Kubernetes PodTemplateSpec). The parts of a SeldonDeployment are shown below:
+
+![inference-graph](./inf-graph.png)
+
+A minimal example for a single model, this time in YAML, is shown below:
+```
+apiVersion: machinelearning.seldon.io/v1alpha2
+kind: SeldonDeployment
+metadata:
+  name: seldon-model
+spec:
+  name: test-deployment
+  predictors:
+  - componentSpecs:
+    - spec:
+        containers:
+        - image: seldonio/mock_classifier:1.0
+    graph:
+      children: []
+      endpoint:
+        type: REST
+      name: classifier
+      type: MODEL
+    name: example
+    replicas: 1
+```
+
+The key components are:
+
+  * A list of Predictors, each with a specification for the number of relicas.
+     * Each defines a graph and its set of deployments. Multiple predictors is useful when you want to split traffic between a main graph and a canary or for other production rollout scenarios.
+  * For each predictor a list of componentSpecs. Each componentSpec is a Kubernetes PodTemplateSpec which Seldon will build into a Kubernetes Deployment. Place here the images from your graph and their requirements, e.g. Volumes, ImagePullSecrets, Resources Requests etc.
+  * A graph specification that describes how your components are joined together.
+
+To understand the inference graph definition in detail see [here](crd/readme.md)
+
+## Next Steps
+
+ * [Jupyter notebooks showing worked examples](../readme.md#quick-start)
+ * Templated Helm Charts:
+   * [Single Model with optional Outlier Detector](../helm-charts/seldon-single-model/README.md)
+   * [AB Test between two models](../helm-charts/seldon-abtest/README.md)
+   * [Multi-Armed Bandit of two models](../helm-charts/seldon-mab/README.md)   
+ * [Integration with other machine learning frameworks](../readme.md#integrations)
+

helm-charts/Makefile

@@ -1,4 +1,4 @@
-CHARTS=seldon-core-crd seldon-core seldon-core-analytics seldon-core-kafka seldon-core-loadtesting
+CHARTS=seldon-core-crd seldon-core seldon-core-analytics seldon-core-kafka seldon-core-loadtesting seldon-single-model seldon-abtest seldon-mab
 
 
 build_all:

helm-charts/seldon-abtest/Chart.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+description: Seldon Core AB test template. Allows you to split traffic between two models.
+keywords:
+- kubernetes
+- machine-learning
+name: seldon-abtest
+sources:
+- https://github.com/SeldonIO/seldon-core
+version: 0.1

helm-charts/seldon-abtest/README.md

@@ -0,0 +1,4 @@
+# AB Test
+
+This chart creates an AB test over two models. You can choose to separate the models into individual Deployment controlled Kubernetes Pods or have them in a single Pod via the separate_pods value.
+

helm-charts/seldon-abtest/templates/ab_test_1pod.json

@@ -0,0 +1,82 @@
+{{- if not .Values.separate_pods }}        
+{
+    "apiVersion": "machinelearning.seldon.io/v1alpha2",
+    "kind": "SeldonDeployment",
+    "metadata": {
+	"labels": {
+	    "app": "seldon"
+	},
+	"name": "seldon-deployment-example"
+    },
+    "spec": {
+	"name": "{{ .Values.name }}",
+{{- if .Values.oauth.key }}        
+        "oauth_key": "{{ .Values.oauth.key }}",
+        "oauth_secret": "{{ .Values.oauth.secret }}",
+{{- end }}
+	"predictors": [
+	    {
+		"name": "fx-market-predictor",
+		"replicas": 1,
+		"componentSpecs": [{
+		    "spec": {
+			"containers": [
+			    {
+				"image": "{{ .Values.modelb.image.name }}",
+				"imagePullPolicy": "IfNotPresent",
+				"name": "{{ .Values.modela.name }}",
+				"resources": {
+				    "requests": {
+					"memory": "1Mi"
+				    }
+				}
+			    },
+			    {
+				"image": "{{ .Values.modelb.image.name }}",
+				"imagePullPolicy": "IfNotPresent",
+				"name": "{{ .Values.modelb.name }}",
+				"resources": {
+				    "requests": {
+					"memory": "1Mi"
+				    }
+				}
+			    }
+			],
+			"terminationGracePeriodSeconds": 20
+				   }
+				   }],
+		"graph": {
+		    "name": "random-ab-test",
+		    "endpoint":{},
+		    "implementation":"RANDOM_ABTEST",
+		    "parameters": [
+			{
+			    "name":"ratioA",
+			    "value":"{{ .Values.traffic_modela_percentage }}",
+			    "type":"FLOAT"
+			}
+		    ],
+		    "children": [
+			{
+			    "name": "{{ .Values.modela.name }}",			    
+			    "endpoint":{
+				"type":"{{ .Values.modela.endpoint }}"
+			    },
+			    "type":"MODEL",
+			    "children":[]
+			},
+			{
+			    "name": "{{ .Values.modelb.name }}",			    
+			    "endpoint":{
+				"type":"{{ .Values.modela.endpoint }}"
+			    },
+			    "type":"MODEL",
+			    "children":[]
+			}   
+		    ]
+		}
+	    }
+	]
+    }
+}
+{{- end }}

helm-charts/seldon-abtest/templates/ab_test_2pods.json

@@ -0,0 +1,95 @@
+{{- if .Values.separate_pods }}
+{
+    "apiVersion": "machinelearning.seldon.io/v1alpha2",
+    "kind": "SeldonDeployment",
+    "metadata": {
+	"labels": {
+	    "app": "seldon"
+	},
+	"name": "seldon-deployment"
+    },
+    "spec": {
+	"name": "{{ .Values.name }}",
+{{- if .Values.oauth.key }}        
+        "oauth_key": "{{ .Values.oauth.key }}",
+        "oauth_secret": "{{ .Values.oauth.secret }}",
+{{- end }}
+	"predictors": [
+	    {
+		"name": "abtest",
+		"replicas": 1,
+		"componentSpecs": [{
+		    "spec": {
+			"containers": [
+			    {
+				"image": "{{ .Values.modelb.image.name }}",
+				"imagePullPolicy": "IfNotPresent",
+				"name": "{{ .Values.modela.name }}",
+				"resources": {
+				    "requests": {
+					"memory": "1Mi"
+				    }
+				}
+			    }],
+			"terminationGracePeriodSeconds": 20
+		    }},
+	        {
+		    "metadata":{
+			"labels":{
+			    "version":"v2"
+			}
+		    },    
+			"spec":{
+			    "containers":[
+			    {
+				"image": "{{ .Values.modelb.image.name }}",
+				"imagePullPolicy": "IfNotPresent",
+				"name": "{{ .Values.modelb.name }}",
+				"resources": {
+				    "requests": {
+					"memory": "1Mi"
+				    }
+				}
+			    }
+			],
+			"terminationGracePeriodSeconds": 20
+				   }
+				   }],
+		"graph": {
+		    "name": "random-ab-test",
+		    "endpoint":{},
+		    "implementation":"RANDOM_ABTEST",
+		    "parameters": [
+			{
+			    "name":"ratioA",
+			    "value":"0.5",
+			    "type":"FLOAT"
+			}
+		    ],
+		    "children": [
+			{
+			    "name": "{{ .Values.modela.name }}",
+			    "endpoint":{
+				"type":"REST"
+			    },
+			    "type":"MODEL",
+			    "children":[]
+			},
+			{
+			    "name": "{{ .Values.modelb.name }}",
+			    "endpoint":{
+				"type":"REST"
+			    },
+			    "type":"MODEL",
+			    "children":[]
+			}   
+		    ]
+		}
+	    }
+	]
+    }
+}
+{{- end }}		
+
+
+

helm-charts/seldon-abtest/values.yaml

@@ -0,0 +1,20 @@
+separate_pods: true
+name: my-abtest
+modela:
+  image:
+    name: seldonio/mock_classifier:1.0
+  endpoint: REST
+  name: classifier-1
+#  resources: { "requests": { "memory": "1Mi" }}
+modelb:
+  image:
+    name: seldonio/mock_classifier:1.0
+  endpoint: REST
+  name: classifier-2
+traffic_modela_percentage: 0.5
+replicas: 1
+# Add oauth key and secret if using the default API Oauth Gateway for ingress
+oauth:
+  key: 
+  secret:
+

helm-charts/seldon-mab/Chart.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+description: Seldon Core Multi-Armed Bandit Solver template. Uses a multi-armed bandit solver to send traffic to the best performing model. You will need to utilize both the predict and send_feedback API methods.
+keywords:
+- kubernetes
+- machine-learning
+name: seldon-mab
+sources:
+- https://github.com/SeldonIO/seldon-core
+version: 0.1

helm-charts/seldon-mab/README.md

@@ -0,0 +1,3 @@
+# Multi-Armed Bandit Over Two Models
+
+This chart provides a multi-armed bandit over two models. The default multi-armed bandit solver is the [epsilon-greedy solver](../../notebooks/epsilon_greedy_gcp.ipynb) provided as part of Seldon Core.