Switch branches/tags
Nothing to show
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
README-training.md Update README-training.md Nov 29, 2018
README.md Update README.md Nov 29, 2018

README.md

PipelineAI Logo

Pull PipelineAI Sample Models

git clone https://github.com/PipelineAI/models

Change into the new models/ directory

cd models

Install PipelineAI

System Requirements

  • 8GB
  • 4 Cores

Requirements

Install PipelineAI CLI

  • Click HERE to install the PipelineAI CLI

Sync Latest Docker Images

pipeline env-registry-sync --tag=1.5.0
  • For GPU images, add --chip=gpu

Deploy Models

Deploy a TensorFlow Model

Inspect Model Directory

Note: This is relative to where you cloned the models repo above.

ls -l ./tensorflow/mnist-v1/model

### EXPECTED OUTPUT ###
...
pipeline_conda_environment.yaml    <-- Required. Sets up the conda environment
pipeline_condarc                   <-- Required, but Empty is OK.  Configure Conda proxy servers (.condarc)
pipeline_modelserver.properties    <-- Required, but Empty is OK.  Configure timeouts and fallbacks
pipeline_invoke_python.py          <-- Required if using Python Runtime (python)
pipeline_invoke_tflite.py          <-- Required if using TensorFlow Lite Runtime (tflite)
pipeline_invoke_tfserving.py       <-- Required if using TensorFlow Serving Runtime (tfserving)
pipeline_setup.sh                  <-- Required, but Empty is OK.  Init script performed upon Docker build.
pipeline_tfserving.properties      <-- Required by TensorFlow Serving. Custom request-batch sizes, etc.
pipeline_tfserving/                <-- Required by TensorFlow Serving. Contains the TF SavedModel
...

Inspect TensorFlow Serving Model

ls -l ./tensorflow/mnist-v1/model/pipeline_tfserving/

### EXPECTED OUTPUT ###
...
0/  <-- TensorFlow Serving finds the latest (highest) version 
...
ls -l ./tensorflow/mnist-v1/model/pipeline_tfserving/0/

### EXPECTED OUTPUT ###
...
saved_model.pb  <-- Graph
variables/      <-- Variables
...

Build the Model into a Runnable Docker Image

  • This command bundles the TensorFlow runtime with the model.
pipeline predict-server-build --model-name=mnist --model-tag=v1tensorflow --model-type=tensorflow --model-runtime=python --model-path=./tensorflow/mnist-v1/model

Notes:

  • --model-path must be relative
  • Start Docker on your machine if you see ERRO[0002] failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: connection refused
  • Add --http-proxy=... and --https-proxy=... if you see CondaHTTPError: HTTP 000 CONNECTION FAILED for url
  • If you have issues, see the comprehensive Troubleshooting section below.
  • --model-type: tensorflow, scikit, python, keras, spark, java, xgboost, pmml, caffe
  • --model-runtime: jvm, tfserving, python, cpp, tensorrt
  • --model-chip: cpu, gpu, tpu
  • For GPU-based models, make sure you specify --model-chip=gpu

Start the Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1tensorflow --predict-memory-limit=2G

Notes:

  • Ignore WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.

  • If you see Unable to find image 'pipelineai/predict-mnist:v1tensorflow' locally docker: Error response from daemon: manifest for pipelineai/predict-mnist:v1tensorflow not found., make sure the Docker image built successfully above.

  • If you see port is already allocated or already in use by container, you already have a container running. List and remove any conflicting containers. For example, docker ps and/or docker rm -f train-mnist-v1.

  • You can change the port(s) by specifying the following: --predict-port=8081, --prometheus-port=9091, --grafana-port=3001.

  • If you change the ports, be sure to change the ports in the examples below to match your new ports.

  • Also, your nginx and prometheus configs will need to be adjusted.

  • In other words, try not to change the ports!

  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

  • If you're having trouble, see our Troubleshooting Guide.

Monitor Runtime Logs

  • Wait for the model runtime to settle...
pipeline predict-server-logs --model-name=mnist --model-tag=v1tensorflow

### EXPECTED OUTPUT ###
...
2017-10-10 03:56:00.695  INFO 121 --- [     run-main-0] i.p.predict.jvm.PredictionServiceMain$   : Started PredictionServiceMain. in 7.566 seconds (JVM running for 20.739)
[debug] 	Thread run-main-0 exited.
[debug] Waiting for thread container-0 to terminate.

Notes:

  • You need to Ctrl-C out of the log viewing before proceeding.

Predict in Any Language

  • Use the REST API to POST a JSON document representing a number.

MNIST 8

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

### Expected Output ###
{
  "variant": "83f05e58mnist-v1tfpythoncpu2286b938-tensorflow-python-cpu",
  "outputs": {
    "classes": [
      7
    ],
    "probabilities": [
      [
        0.011090366169810295,
        0.000009538731319480576,
        0.00006389449117705226,
        0.0006141681806184351,
        0.0037147400435060263,
        0.005028672982007265,
        0.00008352005534106866,
        0.925011157989502,        <== Prediction
        0.0032481630332767963,
        0.05113571882247925
      ]
    ]
  }
}

### FORMATTED OUTPUT ###
Digit  Confidence
=====  ==========
0      0.011090366169810295,
1      0.000009538731319480576,
2      0.00006389449117705226,
3      0.0006141681806184351,
4      0.0037147400435060263,
5      0.005028672982007265,
6      0.00008352005534106866,
7      0.925011157989502,         <== Prediction
8      0.0032481630332767963,
9      0.05113571882247925

Notes:

  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

PipelineCLI Predict

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./tensorflow/mnist-v1/model/pipeline_test_request.json

### EXPECTED OUTPUT ###
{
  "variant": "83f05e58mnist-v1tfpythoncpu2286b938-tensorflow-python-cpu",
  "outputs": {
    "classes": [
      7
    ],
    "probabilities": [
      [
        0.011090366169810295,
        0.000009538731319480576,
        0.00006389449117705226,
        0.0006141681806184351,
        0.0037147400435060263,
        0.005028672982007265,
        0.00008352005534106866,
        0.925011157989502,
        0.0032481630332767963,
        0.05113571882247925
      ]
    ]
  }
}

Inspect pipeline_invoke_python.py

Note: Only the invoke() method is required. Everything else is optional.

cat ./tensorflow/mnist-v1/model/pipeline_invoke_python.py

### EXPECTED OUTPUT ###

import os
import numpy as np
import json
import logging                                                 <== Optional.  Log to console, file, kafka
from pipeline_monitor import prometheus_monitor as monitor     <== Optional.  Monitor runtime metrics
from pipeline_logger import log

import tensorflow as tf
from tensorflow.contrib import predictor

_logger = logging.getLogger('pipeline-logger')
_logger.setLevel(logging.INFO)
_logger_stream_handler = logging.StreamHandler()
_logger_stream_handler.setLevel(logging.INFO)
_logger.addHandler(_logger_stream_handler)

__all__ = ['invoke']                                           <== Optional.  Being a good Python citizen.

_labels = {                                                    <== Optional.  Used for metrics/labels
           'name': 'mnist',
           'tag': 'v1',
           'type': 'tensorflow',
           'runtime': 'python',
           'chip': 'cpu',
          }

def _initialize_upon_import():                                 <== Optional.  Called once upon server startup
    ''' Initialize / Restore Model Object.
    '''
    saved_model_path = './pipeline_tfserving/0'
    return predictor.from_saved_model(saved_model_path)


# This is called unconditionally at *module import time*...
_model = _initialize_upon_import()


@log(labels=_labels, logger=_logger)                           <== Optional.  Sample and compare predictions
def invoke(request):                                           <== Required.  Called on every prediction
    '''Where the magic happens...'''

    with monitor(labels=_labels, name="transform_request"):    <== Optional.  Expose fine-grained metrics
        transformed_request = _transform_request(request)      <== Optional.  Transform input (json) into TensorFlow (tensor)

    with monitor(labels=_labels, name="invoke"):               <== Optional.  Calls _model.predict()
        response = _model(transformed_request)

    with monitor(labels=_labels, name="transform_response"):   <== Optional.  Transform TensorFlow (tensor) into output (json)
        transformed_response = _transform_response(response)

    return transformed_response                                <== Required.  Returns the predicted value(s)


def _transform_request(request):
    request_str = request.decode('utf-8')
    request_json = json.loads(request_str)
    request_np = (np.array(request_json['image'], dtype=np.float32) / 255.0).reshape(1, 28, 28)
    return {"image": request_np}


def _transform_response(response):
    return json.dumps({"classes": response['classes'].tolist(),
                       "probabilities": response['probabilities'].tolist(),
                      })

if __name__ == '__main__':
    with open('./pipeline_test_request.json', 'rb') as fb:
        request_bytes = fb.read()
        response_bytes = invoke(request_bytes)
        print(response_bytes)

Deploy a Scikit-Learn Model

View Prediction Code

cat ./scikit/mnist-v1/model/pipeline_invoke_python.py

Build the SciKit-Learn Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1scikit --model-type=scikit --model-runtime=python --model-path=./scikit/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the SciKit-Learn Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1scikit
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

View the SciKit-Learn Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1scikit

Deploy a PyTorch Model

View Prediction Code

cat ./pytorch/mnist-v1/model/pipeline_invoke_python.py

Build the PyTorch Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1pytorch --model-type=pytorch --model-runtime=python --model-path=./pytorch/mnist-v1/model/ 
  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

Start the PyTorch Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1pytorch
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

View the PyTorch Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1pytorch

Deploy an Xgboost Model

View Prediction Code

cat ./xgboost/mnist-v1/model/pipeline_invoke_python.py

Build the Xgboost Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1xgboost --model-type=xgboost --model-runtime=python --model-path=./xgboost/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the Xgboost Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1xgboost
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

View the Xgboost Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1xgboost

Deploy an MXNet Model

View Prediction Code

cat ./mxnet/mnist-v1/model/pipeline_invoke_python.py

Build the MXNet Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1mxnet --model-type=mxnet --model-runtime=mxnet --model-path=./mxnet/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the MXNet Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1mxnet
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you use nvidia-docker2 with --start-cmd-extra-args="--runtime=nvidia" (or nvidia-docker with --start-cmd=nvidia-docker)

View the MXNet Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1mxnet

Model Predict

Predict with REST API

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

Notes:

  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

Predict with CLI

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./mxnet/mnist-v1/model/pipeline_test_request.json

Deploy a Spark Model

View Prediction Code

cat ./spark/mnist-v1/model/pipeline_invoke_jvm.scala

Build the Spark Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1spark --model-type=spark --model-runtime=jvm --model-path=./spark/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the Spark Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1spark
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the Spark Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1spark

Model Predict

Predict with REST API

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

Notes:

  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

Predict with CLI

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./spark/mnist-v1/model/pipeline_test_request.json

Train Models with PipelineAI

  • Click HERE to Train Models with PipelineAI

PipelineAI Quick Start (CPU, GPU, and TPU)

Train and Deploy your ML and AI Models in the Following Environments: