MLeap serving overhaul (#407)

* Working on core executor for serving MLeap models.

* Add in bundle meta/relaxed select mode.

* Add some hopefully nice Java methods for TransformService.

* Starting to add in specs for everything.

* Basic tests for row/frame transforms.

* Working on gRPC server.

* gRPC server almost ready, just need tests.

* adding more tests for mleap-executor module

* improve coverage of existing tests

* More efficient row streams.

* Nice client/server for gRPC interface.

* Add frame flow to executor.

* Implement frame streaming with gRPC.

* Add timeouts to client.

* add first tests to bundle actor, remove TransformRowRequest from transform service

* remove flow from RowTransformClient

* update timeout method to match proto definition

* grpc server tests with blocking client stub only (for now)

* change trait for before/after each test

* add empty spring boot module

* separate grpc service from proto messages so that they can be re-used in spring boot serving

* separate grpc service from proto messages so that they can be re-used in spring boot serving

* autowire mleap executor in spring boot scoring

* remove unintended change

* make protos work with Spring and ProtobufHttpMessageConverter

* implement get bundle meta in spring boot (+ test for happy case)

* start implementing transform frame

* fix converter on select mode

* add transform frame endpoint (happy case) + test for happy path

* tidy-ups

* completion stage works fine

* add GetBundleMetaResponse to start handling error cases on both get bundle meta & transform frame

* revert changes for error handling

* handle error cases + tests for bundle meta endpoint

* missed change

* error cases for transform frame endpoint

* fix scoring transform error, to return 200 and error info in the response

* turn off logging for tests

* returning the same status code as Springboot for Spring related exceptions

* add dependency for json endpoints

* add json endpoints

* add options check to one of the tests

* add test for json frame

* switch tests to use real client and server

* add akka streams test kit for testing flows

* Fix up response source problem.

* add format to grpc client so that exception is not thrown when creating rowreader/writers

* use binary format only

* add row test so far

* Fix up issues with gRPC streams.

* more tests for row and frame flows

* match 400 status error for both json and protobuf

* Fix row transformer.

* Fix transform frame flow.

* Fix up gRPC server specs.

* Fix up gRPC streams and spec.

* Resource clean up tests for gRPC.

* Make repositories configurable with Typesafe configs.

* Get bootable, running gRPC server.

* Enable docker builds for mleap-grpc-server.

* Get rid of docker settings for now.

* Make sure parallelism parameters get passed through.

* Update usage of timeouts.

* Update to gRPC Akka Streams.

* Reworking executor service.

* Update to executor an gRPC server.

* Fix Spring tests.

* Get rid of unused import.

* Make sure to start/stop services properly.

* Rename inactivity_timeout.

* Working on new model loading/config system.

* Almost have new architecture of executor working.

* Almost done with new executor + testing.

* Executor working with new framework.

* Working client.

* Updating API for gRPC service.

* Refactoring tests.

* Refactor executor tests into testkit.

* Working on tests for row transformer.

* Improving error handling.

* Test all timeouts and delays.

* Get rid of transform timeout.

* gRPC properly wrapping/unwrapping errors.

* Add in RowTransformClient.

* re-work json and proto endpoints to match mleap executor

* fix wrong log

* add tests for all json endpoints except transform

* Add in S3 repository.

* refactor tests so that they work for all protobuf endpoints except transform as well

* start adding transform endpoint tests

* adding more transform endpoint tests

* adding more transform endpoint tests withe select options

* return a more appropriate 201(accepted) for loading a model

* handle transform failures + tests

* handle transform failure when leap frame parsing fails + tests

* adding transform debugging endpoints that don't require encoding the leap frame

* remove no longer required test

* add exception handler for timeouts

* add tests for transforming leap frame directly

* remove annotation not required

* add swagger documentation file

* Working on DB shaded jar.

* Setting up facade jar from databricks runtime.

* MLeap basics are working fine with testkit.

* Add README.

* Working on tests for fat jar.

* MLlibLocal and transitive dependencies are provided.

* Add in XGboost integration to MLeap jar.

* Do not publish databricks runtime fat.

* XGboost is working.

* Add in better READMEs for the assembly JAR.

* Updating xgboost to provided maven central libraries.

* Parity for updated XGboost integration.

* Include XGboost Spark integration in aggregates.

* Get rid of unused submodule.

* Fix up tests for XGBoost with Databricks runtime.

* Update README for assembly.

* Get rid of xgboost predictor Java submodule.

* Add in remaining fields/parameters for XGBoost models.

* Include databricks runtime as an aggregate.

* Do not mark xgboost-spark as provided.

* Fix up dependencies.

* Update bundle protobuf submodule.

* Update to latest proto definitions.

* Fix up specs for gRPC server.

* MLeap serving will serve both gRPC and HTTP services.

* MLeap serving should share executor with spring boot/grpc.

* Make default memory/disk timeout configurable.

* Working on type converter.

* Make configuration parameters optional and fallback to config values.

* Working on executor README.md.

* Better documentation for configuraiton.

* draft of spring-boot readme

* fix compilation issues, remove mleap-serving from travis.sh as it's already running as part of the build

* address failure in protobuf test

* attempt to fix tests again

* test different order

* revert order

* start adding spring boot examples

* add transform endpoint documentation

* fix broken test due to too short timeouts on loading model request

* add prototyping requests in readme

* update to latest spring boot version

* adding assemby details

* retrigger build

* use stable version of sckit-learn

* update spring boot readme to use docker instead

* fix python failures

* check and update documentation

* final touches on documentation

build.sbt

@@ -16,7 +16,14 @@ lazy val tensorflow = MleapProject.tensorflow
 lazy val xgboostRuntime = MleapProject.xgboostRuntime
 lazy val xgboostSpark = MleapProject.xgboostSpark
 lazy val serving = MleapProject.serving
+lazy val executor = MleapProject.executor
+lazy val executorTestKit = MleapProject.executorTestKit
+lazy val executorTests = MleapProject.executorTests
+lazy val repositoryS3 = MleapProject.repositoryS3
+lazy val grpc = MleapProject.grpc
+lazy val grpcServer = MleapProject.grpcServer
 lazy val benchmarks = MleapProject.benchmark
+lazy val springBoot = MleapProject.springBootServing
 lazy val databricksRuntimeFat = MleapProject.databricksRuntimeFat
 lazy val databricksRuntime = MleapProject.databricksRuntime
 lazy val databricksRuntimeTestkit = MleapProject.databricksRuntimeTestkit

mleap-executor-testkit/build.sbt

@@ -0,0 +1,4 @@
+import ml.combust.mleap.{Dependencies, Common}
+
+Common.defaultMleapServingSettings
+Dependencies.executorTestKit

mleap-executor-testkit/src/main/resources/leap_frame/frame.airbnb.json

@@ -0,0 +1,45 @@
+{
+  "schema": {
+    "fields": [{
+      "name": "state",
+      "type": "string"
+    }, {
+      "name": "bathrooms",
+      "type": "double"
+    }, {
+      "name": "square_feet",
+      "type": "double"
+    }, {
+      "name": "bedrooms",
+      "type": "double"
+    }, {
+      "name": "security_deposit",
+      "type": "double"
+    }, {
+      "name": "cleaning_fee",
+      "type": "double"
+    }, {
+      "name": "extra_people",
+      "type": "double"
+    }, {
+      "name": "number_of_reviews",
+      "type": "double"
+    }, {
+      "name": "review_scores_rating",
+      "type": "double"
+    }, {
+      "name": "room_type",
+      "type": "string"
+    }, {
+      "name": "host_is_superhost",
+      "type": "string"
+    }, {
+      "name": "cancellation_policy",
+      "type": "string"
+    }, {
+      "name": "instant_bookable",
+      "type": "string"
+    }]
+  },
+  "rows": [["NY", 2.0, 1250.0, 3.0, 50.0, 30.0, 2.0, 56.0, 90.0, "Entire home/apt", "1.0", "strict", "1.0"]]
+}

mleap-executor-testkit/src/main/scala/ml/combust/mleap/executor/testkit/TestUtil.scala

@@ -0,0 +1,48 @@
+package ml.combust.mleap.executor.testkit
+
+import java.io.File
+import java.net.URI
+
+import ml.combust.bundle.BundleFile
+import ml.combust.bundle.dsl.Bundle
+import ml.combust.mleap.core.types.StructType
+import ml.combust.mleap.runtime.MleapSupport._
+import ml.combust.mleap.runtime.frame.{DefaultLeapFrame, RowTransformer, Transformer}
+import ml.combust.mleap.runtime.serialization.FrameReader
+import resource.managed
+
+object TestUtil {
+  lazy val rfUri: URI = {
+    getClass.getClassLoader.getResource("models/airbnb.model.rf.zip").toURI
+  }
+
+  lazy val lrUri: URI = {
+    getClass.getClassLoader.getResource("models/airbnb.model.lr.zip").toURI
+  }
+
+  lazy val faultyFrame: DefaultLeapFrame = {
+    DefaultLeapFrame(StructType(), Seq())
+  }
+
+  lazy val frame: DefaultLeapFrame = {
+    FrameReader().read(new File(getClass.getClassLoader.getResource("leap_frame/frame.airbnb.json").getFile)).get
+  }
+
+  lazy val rfBundle : Bundle[Transformer] = {
+    val bundle = (for(bundle <- managed(BundleFile(new File(rfUri.getPath)))) yield {
+      bundle.loadMleapBundle().get
+    }).tried.get
+
+    bundle
+  }
+
+  lazy val lrBundle : Bundle[Transformer] = {
+    val bundle = (for(bundle <- managed(BundleFile(new File(lrUri.getPath)))) yield {
+      bundle.loadMleapBundle().get
+    }).tried.get
+
+    bundle
+  }
+
+  lazy val rfRowTransformer: RowTransformer = rfBundle.root.transform(RowTransformer(frame.schema)).get
+}