Switch branches/tags
Nothing to show
Find file History
Permalink
..
Failed to load latest commit information.
README-training.md Update README-training.md Nov 7, 2018
README.md Update README.md Oct 25, 2018

README.md

PipelineAI Logo

Pull PipelineAI Sample Models

git clone https://github.com/PipelineAI/models

Change into the new models/ directory

cd models

Install PipelineAI

System Requirements

  • 8GB
  • 4 Cores

Requirements

Install PipelineAI CLI

  • Click HERE to install the PipelineAI CLI

Sync Latest Docker Images

pipeline env-registry-sync --tag=1.5.0
  • For GPU images, add --chip=gpu

Deploy Models

Deploy a TensorFlow Model

Inspect Model Directory

Note: This is relative to where you cloned the models repo above.

ls -l ./tensorflow/mnist-v1/model

### EXPECTED OUTPUT ###
...
pipeline_conda_environment.yaml    <-- Required. Sets up the conda environment
pipeline_condarc                   <-- Required, but Empty is OK.  Configure Conda proxy servers (.condarc)
pipeline_modelserver.properties    <-- Required, but Empty is OK.  Configure timeouts and fallbacks
pipeline_invoke_python.py          <-- Required if using Python Runtime (python)
pipeline_invoke_tflite.py          <-- Required if using TensorFlow Lite Runtime (tflite)
pipeline_invoke_tfserving.py       <-- Required if using TensorFlow Serving Runtime (tfserving)
pipeline_setup.sh                  <-- Required, but Empty is OK.  Init script performed upon Docker build.
pipeline_tfserving.properties      <-- Required by TensorFlow Serving. Custom request-batch sizes, etc.
pipeline_tfserving/                <-- Required by TensorFlow Serving. Contains the TF SavedModel
...

Inspect TensorFlow Serving Model

ls -l ./tensorflow/mnist-v1/model/pipeline_tfserving/

### EXPECTED OUTPUT ###
...
0/  <-- TensorFlow Serving finds the latest (highest) version 
...
ls -l ./tensorflow/mnist-v1/model/pipeline_tfserving/0/

### EXPECTED OUTPUT ###
...
saved_model.pb  <-- Graph
variables/      <-- Variables
...

Build the Model into a Runnable Docker Image

  • This command bundles the TensorFlow runtime with the model.
pipeline predict-server-build --model-name=mnist --model-tag=v1tensorflow --model-type=tensorflow --model-runtime=python --model-path=./tensorflow/mnist-v1/model

Notes:

  • --model-path must be relative
  • Start Docker on your machine if you see ERRO[0002] failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: connection refused
  • Add --http-proxy=... and --https-proxy=... if you see CondaHTTPError: HTTP 000 CONNECTION FAILED for url
  • If you have issues, see the comprehensive Troubleshooting section below.
  • --model-type: tensorflow, scikit, python, keras, spark, java, xgboost, pmml, caffe
  • --model-runtime: jvm, tfserving, python, cpp, tensorrt
  • --model-chip: cpu, gpu, tpu
  • For GPU-based models, make sure you specify --model-chip=gpu

Start the Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1tensorflow --memory-limit=2G

Notes:

  • Ignore WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • If you see Unable to find image 'pipelineai/predict-mnist:v1tensorflow' locally docker: Error response from daemon: manifest for pipelineai/predict-mnist:v1tensorflow not found., make sure the Docker image built successfully above.
  • If you see port is already allocated or already in use by container, you already have a container running. List and remove any conflicting containers. For example, docker ps and/or docker rm -f train-mnist-v1.
  • You can change the port(s) by specifying the following: --predict-port=8081, --prometheus-port=9091, --grafana-port=3001.
  • If you change the ports, be sure to change the ports in the examples below to match your new ports.
  • Also, your nginx and prometheus configs will need to be adjusted.
  • In other words, try not to change the ports!
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!
  • If you're having trouble, see our Troubleshooting Guide.

Monitor Runtime Logs

  • Wait for the model runtime to settle...
pipeline predict-server-logs --model-name=mnist --model-tag=v1tensorflow

### EXPECTED OUTPUT ###
...
2017-10-10 03:56:00.695  INFO 121 --- [     run-main-0] i.p.predict.jvm.PredictionServiceMain$   : Started PredictionServiceMain. in 7.566 seconds (JVM running for 20.739)
[debug] 	Thread run-main-0 exited.
[debug] Waiting for thread container-0 to terminate.

Notes:

  • You need to Ctrl-C out of the log viewing before proceeding.

Predict in Any Language

  • Use the REST API to POST a JSON document representing a number.
http://localhost:8080

PipelineAI REST API

MNIST 8

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

### Expected Output ###
{
  "variant": "83f05e58mnist-v1tfpythoncpu2286b938-tensorflow-python-cpu",
  "outputs": {
    "classes": [
      7
    ],
    "probabilities": [
      [
        0.011090366169810295,
        0.000009538731319480576,
        0.00006389449117705226,
        0.0006141681806184351,
        0.0037147400435060263,
        0.005028672982007265,
        0.00008352005534106866,
        0.925011157989502,        <== Prediction
        0.0032481630332767963,
        0.05113571882247925
      ]
    ]
  }
}

### FORMATTED OUTPUT ###
Digit  Confidence
=====  ==========
0      0.011090366169810295,
1      0.000009538731319480576,
2      0.00006389449117705226,
3      0.0006141681806184351,
4      0.0037147400435060263,
5      0.005028672982007265,
6      0.00008352005534106866,
7      0.925011157989502,         <== Prediction
8      0.0032481630332767963,
9      0.05113571882247925

Notes:

  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

PipelineCLI Predict

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./tensorflow/mnist-v1/model/pipeline_test_request.json

### EXPECTED OUTPUT ###
{
  "variant": "83f05e58mnist-v1tfpythoncpu2286b938-tensorflow-python-cpu",
  "outputs": {
    "classes": [
      7
    ],
    "probabilities": [
      [
        0.011090366169810295,
        0.000009538731319480576,
        0.00006389449117705226,
        0.0006141681806184351,
        0.0037147400435060263,
        0.005028672982007265,
        0.00008352005534106866,
        0.925011157989502,
        0.0032481630332767963,
        0.05113571882247925
      ]
    ]
  }
}

Inspect pipeline_invoke_python.py

Note: Only the invoke() method is required. Everything else is optional.

cat ./tensorflow/mnist-v1/model/pipeline_invoke_python.py

### EXPECTED OUTPUT ###

import os
import numpy as np
import json
import logging                                                 <== Optional.  Log to console, file, kafka
from pipeline_monitor import prometheus_monitor as monitor     <== Optional.  Monitor runtime metrics
from pipeline_logger import log

import tensorflow as tf
from tensorflow.contrib import predictor

_logger = logging.getLogger('pipeline-logger')
_logger.setLevel(logging.INFO)
_logger_stream_handler = logging.StreamHandler()
_logger_stream_handler.setLevel(logging.INFO)
_logger.addHandler(_logger_stream_handler)

__all__ = ['invoke']                                           <== Optional.  Being a good Python citizen.

_labels = {                                                    <== Optional.  Used for metrics/labels
           'name': 'mnist',
           'tag': 'v1',
           'type': 'tensorflow',
           'runtime': 'python',
           'chip': 'cpu',
          }

def _initialize_upon_import():                                 <== Optional.  Called once upon server startup
    ''' Initialize / Restore Model Object.
    '''
    saved_model_path = './pipeline_tfserving/0'
    return predictor.from_saved_model(saved_model_path)


# This is called unconditionally at *module import time*...
_model = _initialize_upon_import()


@log(labels=_labels, logger=_logger)                           <== Optional.  Sample and compare predictions
def invoke(request):                                           <== Required.  Called on every prediction
    '''Where the magic happens...'''

    with monitor(labels=_labels, name="transform_request"):    <== Optional.  Expose fine-grained metrics
        transformed_request = _transform_request(request)      <== Optional.  Transform input (json) into TensorFlow (tensor)

    with monitor(labels=_labels, name="invoke"):               <== Optional.  Calls _model.predict()
        response = _model(transformed_request)

    with monitor(labels=_labels, name="transform_response"):   <== Optional.  Transform TensorFlow (tensor) into output (json)
        transformed_response = _transform_response(response)

    return transformed_response                                <== Required.  Returns the predicted value(s)


def _transform_request(request):
    request_str = request.decode('utf-8')
    request_json = json.loads(request_str)
    request_np = (np.array(request_json['image'], dtype=np.float32) / 255.0).reshape(1, 28, 28)
    return {"image": request_np}


def _transform_response(response):
    return json.dumps({"classes": response['classes'].tolist(),
                       "probabilities": response['probabilities'].tolist(),
                      })

if __name__ == '__main__':
    with open('./pipeline_test_request.json', 'rb') as fb:
        request_bytes = fb.read()
        response_bytes = invoke(request_bytes)
        print(response_bytes)

Deploy a Scikit-Learn Model

View Prediction Code

cat ./scikit/mnist-v1/model/pipeline_invoke_python.py

Build the SciKit-Learn Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1scikit --model-type=scikit --model-runtime=python --model-path=./scikit/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the SciKit-Learn Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1scikit
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the SciKit-Learn Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1scikit

Deploy a PyTorch Model

View Prediction Code

cat ./pytorch/mnist-v1/model/pipeline_invoke_python.py

Build the PyTorch Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1pytorch --model-type=pytorch --model-runtime=python --model-path=./pytorch/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the PyTorch Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1pytorch
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the PyTorch Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1pytorch

Deploy an Xgboost Model

View Prediction Code

cat ./xgboost/mnist-v1/model/pipeline_invoke_python.py

Build the Xgboost Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1xgboost --model-type=xgboost --model-runtime=python --model-path=./xgboost/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the Xgboost Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1xgboost
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the Xgboost Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1xgboost

Deploy an MXNet Model

View Prediction Code

cat ./mxnet/mnist-v1/model/pipeline_invoke_python.py

Build the MXNet Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1mxnet --model-type=mxnet --model-runtime=mxnet --model-path=./mxnet/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the MXNet Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1mxnet
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the MXNet Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1mxnet

Model Predict

Predict with REST API

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

Notes:

  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

Predict with CLI

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./mxnet/mnist-v1/model/pipeline_test_request.json

Deploy a Spark Model

View Prediction Code

cat ./spark/mnist-v1/model/pipeline_invoke_jvm.scala

Build the Spark Model Server

pipeline predict-server-build --model-name=mnist --model-tag=v1spark --model-type=spark --model-runtime=jvm --model-path=./spark/mnist-v1/model/ 
  • For GPU-based models, make sure you specify --model-chip=gpu - and make sure you have nvidia-docker installed!

Start the Spark Model Server

pipeline predict-server-start --model-name=mnist --model-tag=v1spark
  • Ignore the following warning: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  • For GPU-based models, make sure you specify --start-cmd=nvidia-docker - and make sure you have nvidia-docker installed!

View the Spark Model Server Logs

pipeline predict-server-logs --model-name=mnist --model-tag=v1spark

Model Predict

Predict with REST API

curl -X POST -H "Content-Type: application/json" \
  -d '{"image": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,10,150,238,254,255,237,150,150,225,161,221,203,18,0,0,0,0,0,0,0,0,0,0,0,0,0,0,66,146,253,253,253,253,253,253,253,253,253,253,253,173,0,0,0,0,0,0,0,0,0,0,0,0,0,140,251,253,253,178,114,114,114,114,114,114,167,253,253,154,0,0,0,0,0,0,0,0,0,0,0,12,84,240,253,248,170,28,0,0,0,0,0,0,90,253,250,90,0,0,0,0,0,0,0,0,0,0,10,129,226,253,235,128,0,0,0,0,0,0,0,8,188,253,190,0,0,0,0,0,0,0,0,0,0,0,56,250,253,246,98,0,0,0,0,0,0,0,0,76,243,234,100,0,0,0,0,0,0,0,0,0,0,0,185,253,248,44,0,0,0,0,0,0,0,0,34,245,253,95,0,0,0,0,0,0,0,0,0,0,0,0,69,187,87,0,0,0,0,0,0,0,0,22,164,253,223,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,55,247,253,85,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,230,253,184,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,145,253,241,43,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,57,250,253,206,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,135,253,253,40,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,105,251,248,108,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,68,226,253,180,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,233,253,205,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,189,253,240,72,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,110,253,253,109,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,47,242,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,43,198,228,24,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]}' \
  http://localhost:8080 \
  -w "\n\n"

Notes:

  • You may see 502 Bad Gateway or '{"results":["Fallback!"]}' if you predict too quickly. Let the server settle a bit - and try again.
  • You will likely see Fallback! on the first successful invocation. This is GOOD! This means your timeouts are working. Check out the PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties.
  • If you continue to see Fallback! even after a minute or two, you may need to increase the value of PIPELINE_MODEL_SERVER_TIMEOUT_MILLISECONDS in pipeline_modelserver.properties. (This is rare as the default is 5000 milliseconds, but it may happen.)
  • Instead of localhost, you may need to use 192.168.99.100 or another IP/Host that maps to your local Docker host. This usually happens when using Docker Quick Terminal on Windows 7.
  • If you're having trouble, see our Troubleshooting Guide.

Predict with CLI

pipeline predict-server-test --endpoint-url=http://localhost:8080/invoke --test-request-path=./spark/mnist-v1/model/pipeline_test_request.json

Train Models with PipelineAI

  • Click HERE to Train Models with PipelineAI

PipelineAI Quick Start (CPU, GPU, and TPU)

Train and Deploy your ML and AI Models in the Following Environments: