Skip to content
Branch: master
Find file History
bamurtaugh and CESARDELATORRE Migration/v1.3.1 (#597)
* Add anomaly detection example to solution

* Updated label/score printing for anomaly detect

With ML.NET v1.3.0, fixed issue where Predicted Label was always true. No longer need "hack" of comparing score to 0.2

* Update build props nuget versions

ML.NET v1.3.1, ML Preview 0.15.0

* Renamed solution to match v1.3.1

* Update C# readmes to v1.3.1

* Update F# E2E readme

* Update F# getting started readmes

Change to v1.3.1

* Rename F# solution to v1.3.1

* Update to preview v0.15.1

* Changed to ML from MLPreview

Update TimeSeries to v1.3.1 instead of preview

* Update timeseries from preview to regular v1.3.1

* Change TimeSeries from preview to regular v1.3.1

* Update TensorFlow from Preview to regular v1.3.1

* Update TensorFlow from preview to regular v1.3.1
Latest commit 2feb479 Aug 6, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
src/Scalable.WebAPI Migration/v1.2.0 (#570) Jul 19, 2019 Migration/v1.3.1 (#597) Aug 6, 2019
ScalableMLModelOnWebAPI-IntegrationPkg.sln Added sample for 'net core integration pkg' (#394) Apr 26, 2019

ASP.NET Core WebAPI sample optimized for scalability and performance when running/scoring an ML.NET model (Using the new '.NET Core Integration Package')

ML.NET version Status App Type Data type Scenario ML Task Algorithms
v1.3.1 Up-to-date ASP.NET Core 2.2 WebAPI Single data sample Sentiment Analysis Binary classification Linear Classification

This posts explains how to optimize your code when running an ML.NET model on an ASP.NET Core WebAPI service. The code would be very similar when running it on an ASP.NET Core MVC or Razor web app, too.

This code has been very much simplified by using a new '.NET Core Integration Package' that the .NET team has created (in experimental state) which implements all the 'plumbing' for doing object pooling for the PredictionEngine.


The goal is to be able to make predictions with an ML.NET model while optimizing the executions by sharing the needed ML.NET scoring objects across Http requests and implementing very simple code to be used by the user when predicting, like the following line of code that you could write on any ASP.NET Core controller's method or custom service class:

SamplePrediction prediction = _predictionEnginePool.Predict(sampleData);

As simple as a single line. The object _predictionEnginePool will be injected in the controller's constructor or into your custom class.

Internally, it is optimized so the object dependencies are cached and shared across Http requests with minimum overhead when creating those objects.


The problem running/scoring an ML.NET model in multi-threaded applications comes when you want to do single predictions with the PredictionEngine object and you want to cache that object (i.e. as Singleton) so it is being reused by multiple Http requests (therefore it would be accessed by multiple threads). That's is a problem because the Prediction Engine is not thread-safe (ML.NET issue, Nov 2018).


Use the new '.NET Core Integration Package' that implements Object Pooling of PredictionEngine objects for you

'.NET Core Integration Package' NuGet feed

NuGet package name: Microsoft.Extensions.ML

Preview NuGet feed at MyGet:

Basically, with this component, you inject/use the PredictionEngine object pooling in a single line in your Startup.cs, like the following:

services.AddPredictionEnginePool<SampleObservation, SamplePrediction>()

Then you just need to call the Predict() function from the injected PredictionEnginePool, like the following code you can implement on any controller:

//Predict sentiment
SamplePrediction prediction = _predictionEnginePool.Predict(sampleData);

It is that simple.

For a much more detailed explanation of a PredictionEngine Object Pool comparable to the implementation done in the new '.NET Core Integration Package', including design diagrams, read the following blog post:

Detailed Blog Post for further background documentation:

How to optimize and run ML.NET models on scalable ASP.NET Core WebAPIs or web apps


You can’t perform that action at this time.