Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix typos in docs #241

Merged
merged 1 commit into from
Oct 1, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Each metric has the following key value pairs for further filtering which will b
* predictor_version
* This will be derived from the predictor metadata labels
* model_name
* model_image
* model_image
* model_image


Expand All @@ -39,8 +39,8 @@ helm install seldon-core-analytics --name seldon-core-analytics \

The available parameters are:

* ```grafana_prom_admin_password``` : The admin user grafana password to use.
* ```persistence.enabled``` : Whether prometheus persistence is enabled.
* ```grafana_prom_admin_password``` : The admin user Grafana password to use.
* ```persistence.enabled``` : Whether Prometheus persistence is enabled.

Once running you can expose the Grafana dashboard with:

Expand Down
30 changes: 15 additions & 15 deletions docs/articles/openshift_s2i.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Using Openshift Source-to-Image to facilitate Machine Learning Deployment

Seldon aims to help oraganisations put their data science projects into production so they can decrease the time to get return on investment. By helping data scientists take their data science models and place them into production, scale them, get analytics and modify them Seldon allows data scientists to bridge the gap from development to production and use current dev-ops best practices in machine learning. Our core products run on top of Kubernetes and can be deployed on-cloud on on-premise. Integrating with enterprise ready Kubernetes distributions such as Openshift allows us to provide a solid foundation in which to supply our products for use in demanding verticals such as the FinTech sector.
Seldon aims to help organisations put their data science projects into production so they can decrease the time to get return on investment. By helping data scientists take their data science models and place them into production, scale them, get analytics and modify them Seldon allows data scientists to bridge the gap from development to production and use current dev-ops best practices in machine learning. Our core products run on top of Kubernetes and can be deployed on-cloud on on-premise. Integrating with enterprise ready Kubernetes distributions such as Openshift allows us to provide a solid foundation in which to supply our products for use in demanding verticals such as the FinTech sector.

[Seldon-Core](https://github.com/SeldonIO/seldon-core) is an open source project that provides scalable machine learning deployment running on [Kubernetes](https://kubernetes.io/). One of Seldon-Core’s goals is to allow data scientists to continue to construct their training and inference components using any of the many available machine learning toolkits, be that python based (e.g., TensorFlow, sklearn), R or Java (e.g., Spark, H2O) amongst many popular options. Seldon-Core will then allow them easily to package and run their runtime prediction modules on Kubernetes. To achieve this goal we need to make it easy for data scientists to take their source code and package it as a Docker-formatted container in the correct form such that it can be managed as part of a runtime microservice graph on Kubernetes by Seldon-Core. For this we utilize Openshift’s Source-to-Image open source library to allow any code to be packaged in the correct format with minimal requirements from the data scientist.

# Seldon-Core Overview
Seldon-core provides scalebale machine learning deployments running on Kubernetes. To deploy their models data scientists follow the steps as shown below:
Seldon-core provides scalable machine learning deployments running on Kubernetes. To deploy their models data scientists follow the steps as shown below:

![API](../deploy.png)

1. Package their runtime model as a Docker-formatted image
1. Describe their runtime graph as a kubernetes resource
1. Deploy to kubernetes using standard tools such as kubectl, Helm, ksonnet.
1. Describe their runtime graph as a Kubernetes resource
1. Deploy to Kubernetes using standard tools such as kubectl, Helm, ksonnet.

Once running their deployment can be updated as new image releases are created for the runtime model as well as updates to the runtime graph.

Expand All @@ -26,7 +26,7 @@ The types of component you can create can include:
* Combiners - e.g., Model ensemblers
* Transformers - e.g., Feature normalization, Outlier detection, concept drift

As the above diagram shows these need to be fitted into the microservice API of seldon-core either as REST or gRPC services.
As the above diagram shows these need to be fitted into the microservice API of seldon-core either as REST or gRPC services.

# Source-to-Image integration
To integrate a component into seldon-core the data scientist needs to accomplish two things:
Expand All @@ -52,10 +52,10 @@ class MyModel(object):
"""
Model template. You can load your model parameters in __init__ from a location accessible at runtime
"""

def __init__(self):
"""
Add any initialization parameters. These will be passed at runtime from the graph definition parameters defined in your seldondeployment kubernetes resource manifest.
Add any initialization parameters. These will be passed at runtime from the graph definition parameters defined in your seldondeployment Kubernetes resource manifest.
"""
print("Initializing")

Expand All @@ -76,9 +76,9 @@ class MyModel(object):
* The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
* Any required initialization can be put inside the class init method.

An optional requirements.txt can detail any software dependencies the code requires.
An optional requirements.txt can detail any software dependencies the code requires.

To allow the s2i builder image to correctly package the component the data scinetist needs to provide a few environment variables either in an .s2i/environment file in the source code folder or on the command line. An example is:
To allow the s2i builder image to correctly package the component the data scientist needs to provide a few environment variables either in an .s2i/environment file in the source code folder or on the command line. An example is:

```bash
MODEL_NAME=MyModel
Expand All @@ -94,15 +94,15 @@ s2i build <src-folder> seldonio/seldon-core-s2i-python2 <my-image-name>
```

## R
R is a popular statistical langiage which provides many machine learning related packages.
R is a popular statistical language which provides many machine learning related packages.

To use the seldon s2i builder image to package an R model the requirements are:

* An R file which provides an S3 class for your model via an ```initialise_seldon``` function and that has appropriate generics for the component, e.g. predict for a model.
* An optional install.R to be run to install any libraries needed
* .s2i/environment - model definitions used by the s2i builder to correctly wrap your model

The data scientist's source code should contain an R file which defines an S3 class for their model. For example,
The data scientist's source code should contain an R file which defines an S3 class for their model. For example,

```R
library(methods)
Expand Down Expand Up @@ -140,9 +140,9 @@ API_TYPE=REST
SERVICE_TYPE=MODEL
```

These values can also be provided in an .s2i/environment file with the source code or overriden on the command line when building the image.
These values can also be provided in an .s2i/environment file with the source code or overridden on the command line when building the image.

Once these steps are done we can use ```s2i build``` to create the Docker-formatted image from the source code.
Once these steps are done we can use ```s2i build``` to create the Docker-formatted image from the source code.

```bash
s2i build <git-repo> seldonio/seldon-core-s2i-r <my-image-name>
Expand All @@ -158,14 +158,14 @@ s2i build https://github.com/seldonio/seldon-core.git --context-dir=wrappers/s2i
## Java
There are several popular machine learning libraries in Java including Spark, H2O and DL4J. Seldon-core also provides builder images for Java. To accomplish this we provide a Java library seldon-core-wrappers that can be included in a Maven Spring project to allow a Java component to be easily wrapped.

To use teh Seldon-Core s2i builder image to package a Java model the data scientist will need:
To use the Seldon-Core s2i builder image to package a Java model the data scientist will need:

* A Maven project that depends on the ```io.seldon.wrapper``` library
* A Spring Boot configuration class
* A class that implements ```io.seldon.wrapper.SeldonPredictionService``` for the type of component you are creating
* An optional .s2i/environment - model definitions used by the s2i builder to correctly wrap your model

More details can be found in the seldon-core docs
More details can be found in the seldon-core docs.

# Summary

Expand Down
2 changes: 1 addition & 1 deletion docs/articles/release-0.2.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ spec:

We have an external contribution by @SachinVarghese that provides an initial Seldon Core wrapper for NodeJS allowing you to take advantage of the emerging machine learning tools within the Javascript ecosystem. Thanks Sachin!

An example notebook for an [MNIST Tensorflow model](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/nodejs_tensorflow/nodejs_tensorflow.ipynb) is provided which has the following javascript inference code:
An example notebook for an [MNIST Tensorflow model](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/nodejs_tensorflow/nodejs_tensorflow.ipynb) is provided which has the following Javascript inference code:

```
const tf = require("@tensorflow/tfjs");
Expand Down
10 changes: 5 additions & 5 deletions docs/benchmarking.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Seldon-core Benchmarking

This page is a work in progress to provide benchmarking stats for seldon-core. Please add further ideas and suggestions as an issue.
This page is a work in progress to provide benchmarking stats for seldon-core. Please add further ideas and suggestions as an issue.

## Goals

Expand All @@ -11,7 +11,7 @@ This page is a work in progress to provide benchmarking stats for seldon-core.
## Components

* We use [locust](https://locust.io/) as our benchmarking tool.
* We use Google Cloud Platform for the infrastructure to run kubernetes.
* We use Google Cloud Platform for the infrastructure to run Kubernetes.


# Tests
Expand All @@ -22,11 +22,11 @@ To gauge the maximum throughput we will:
* Call the seldon engine component directly thereby ignoring the additional latency that would be introduced by an external reverse proxy (Ambassador) or using the built in seldon API Front-End Oauth2 component.
* Utilize a "stub" model that does nothing but return a hard-wired result from inside the engine.

This test will illustrate the maximum number of requests that can be pushed through seldon-core engine (which controls the request-response flow) as well as the added latency for the processing of REST and gRPC requests, e.g. serialization / deserialization.
This test will illustrate the maximum number of requests that can be pushed through seldon-core engine (which controls the request-response flow) as well as the added latency for the processing of REST and gRPC requests, e.g. serialization/deserialization.

We will use cordened off kubernetes nodes running locust so the latency from node to node prediction calls on GCP will also be part of the returned statistics.
We will use cordoned off Kubernetes nodes running locust so the latency from node to node prediction calls on GCP will also be part of the returned statistics.

A [notebook](https://github.com/SeldonIO/seldon-core/blob/master/notebooks/benchmark_simple_model.ipynb) provides the end to end test for reproducability.
A [notebook](https://github.com/SeldonIO/seldon-core/blob/master/notebooks/benchmark_simple_model.ipynb) provides the end to end test for reproducibility.

We use:

Expand Down
6 changes: 3 additions & 3 deletions docs/challenges.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Machine Learning deployment has many challenges which Seldon Core's goals are to solve, these include:

* Allow a wide range of ML modeling tools to be easily deployed, e.g. Python, R, Spark, and propritary models
* Allow a wide range of ML modelling tools to be easily deployed, e.g. Python, R, Spark, and proprietary models
* Launch ML runtime graphs, scale up/down, perform rolling updates
* Run health checks and ensure recovery of failed components
* Infrastructure optimization for ML
Expand All @@ -17,11 +17,11 @@ Machine Learning deployment has many challenges which Seldon Core's goals are to
* Asynchronous
* Batch
* Allow Auditing and clear versioning
* Integrate into Continuous Integration (CI)
* Integrate into Continuous Integration (CI)
* Allow Continuous Deployment (CD)
* Provide Monitoring
* Base metrics: Accuracy, request latency and throughput
* Complex metrics:
* Complex metrics:
* Concept drift
* Bias detection
* Outlier detection
Expand Down
12 changes: 6 additions & 6 deletions docs/crd/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ message SeldonDeployment {
}
```

The core deployment spec consists of a set of ```predictors```. Each predictor represents a seperate runtime serving graph. The set of predictors will serve request as controlled by a load balancer. At present the share of traffic will be in relation to the number of replicas each predictor has. A use case for two predictors would be a main deployment and a canary, with the main deployment having 9 replicas and the canary 1, so the canary receives 10% of the overall traffic. Each predictor will be a seperately set of managed deployments with Kubernetes so it is safe to add and remove predictors without affecting existing predictors.
The core deployment spec consists of a set of ```predictors```. Each predictor represents a separate runtime serving graph. The set of predictors will serve request as controlled by a load balancer. At present the share of traffic will be in relation to the number of replicas each predictor has. A use case for two predictors would be a main deployment and a canary, with the main deployment having 9 replicas and the canary 1, so the canary receives 10% of the overall traffic. Each predictor will be a separately set of managed deployments with Kubernetes so it is safe to add and remove predictors without affecting existing predictors.

To allow an OAuth API to be provisioned you should specify an OAuth key and secret. If you are using Ambassador you will not need this as you can plug in your own external authentication using Ambassador.

Expand All @@ -44,7 +44,7 @@ For each predictor you should at a minimum specify:

* A unique name
* A PredictiveUnit graph that presents the tree of components to deploy.
* One or more componentSpecs which describes the set of images for parts of your container graph that will be instigated as microservice containers. These containers will have been wrapped to work within the [internal API](../reference/internal-api.md). This component spec is a standard [PodTemplateSpec](https://kubernetes.io/docs/api-reference/extensions/v1beta1/definitions/#_v1_podtemplatespec). For complex grahs you can decide to use several componentSpecs so as to separate your components into separate Pods each with their own resource requirements.
* One or more componentSpecs which describes the set of images for parts of your container graph that will be instigated as microservice containers. These containers will have been wrapped to work within the [internal API](../reference/internal-api.md). This component spec is a standard [PodTemplateSpec](https://kubernetes.io/docs/api-reference/extensions/v1beta1/definitions/#_v1_podtemplatespec). For complex graphs you can decide to use several componentSpecs so as to separate your components into separate Pods each with their own resource requirements.
* If you leave the ports empty for each container they will be added automatically and matched to the ports in the graph specification. If you decide to specify the ports manually they should match the port specified for the matching component in the graph specification.
* the number of replicas of this predictor to deploy

Expand All @@ -56,20 +56,20 @@ message PredictorSpec {
optional int32 replicas = 4; // The number of replicas of the predictor to create.
map<string,string> annotations = 5; // Arbitrary annotations.
optional k8s.io.api.core.v1.ResourceRequirements engineResources = 6; // Optional set of resources for the Seldon engine which is added to each Predictor graph to manage the request/response flow
map<string,string> labels = 7; // labels to be attached to entry deplyment for this predictor
map<string,string> labels = 7; // labels to be attached to entry deployment for this predictor
}

```

The predictive unit graph is a tree. Each node is of a particular type. If the implementation is not specified then a microservice is assumed and you must define a matching named container within the componentSpec above. Each type of PredictiveUnit has a standard set of methods it is expected to manage, see [here](../reference/seldon-deployment.md).
The predictive unit graph is a tree. Each node is of a particular type. If the implementation is not specified then a microservice is assumed and you must define a matching named container within the componentSpec above. Each type of PredictiveUnit has a standard set of methods it is expected to manage, see [here](../reference/seldon-deployment.md).

For each node in the graph:

* A unique name. If the node describes a microservice then it must match a named container with the componentSpec.
* The children nodes.
* The type of the predictive unit : MODEL, ROUTER, COMBINER, TRANSFORMER or OUTPUT_TRANSFORMER.
* The implementation. This can be left blank if it will be a microserice as this is the default otherwise choose from the available appropriate implementations provided internally.
* Methods. This can be left blank if you wish to follow the standard methods for your PredictiveNode type : see [here](../reference/seldon-deployment.md).
* The implementation. This can be left blank if it will be a microservice as this is the default otherwise choose from the available appropriate implementations provided internally.
* Methods. This can be left blank if you wish to follow the standard methods for your PredictiveNode type : see [here](../reference/seldon-deployment.md).
* Endpoint. In here you should minimally if this a microservice specify whether the PredictiveUnit will use REST or gRPC. Ports will be defined automatically if not specified.
* Parameters. Specify any parameters you wish to pass to the PredictiveUnit. These will be passed in an environment variable called PREDICTIVE_UNIT_PARAMETERS as a JSON list.

Expand Down
6 changes: 3 additions & 3 deletions docs/deploying.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Deployment

You can manage your deployments via the standard kubernetes CLI kubectl, e.g.
You can manage your deployments via the standard Kubernetes CLI kubectl, e.g.

```bash
kubectl apply -f my_ml_deployment.yaml
Expand All @@ -14,8 +14,8 @@ For production settings you will want to incorporate your ML infrastructure and
The pipeline consists of

* A model code repo (in Git) where training and runtime ML components are stored
* A continuuous integration pipeline that will train and test the model and wrap it (using Seldon built-in Wrappers or custome wrappers)
* A continuous integration pipeline that will train and test the model and wrap it (using Seldon built-in Wrappers or custom wrappers)
* An image repository where the final runtime inference model image is stored.
* A git repo for the infrastructure to store the ML deployment graph described as a SeldonDeployment
* Some tool to either monitor the infrastructure repo and apply to the production kubernetes changes or a tool to allow dev ops to push updated infrastructure manually.
* Some tool to either monitor the infrastructure repo and apply to the production Kubernetes changes or a tool to allow dev ops to push updated infrastructure manually.

Loading