Skip to content

Commit 8b93f03

Browse files
ericstjericstj
committed
Add a doc with information about components and dependencies
1 parent b28b6d4 commit 8b93f03

File tree

1 file changed

+91
-0
lines changed

1 file changed

+91
-0
lines changed

‎docs/project-docs/components-and-dependencies.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
ML.NET is a modular system where using it involves building a pipeline from data loaders, trainers/estimators (in case of training), transformers (in case of inferencing), and various data structures to facilitate the building of pipelines and representing data. The core of ML.NET – the Microsoft.ML has no external dependencies. It's largely managed code.
2+
3+
4+
5+
The core ML.NET does have a helper native math library CPUMath - which is only used on .NETFramework. Net6 and later have a managed implementation using intrinsics/TensorPrimitives and do not require the native build of CPUMath.
6+
7+
8+
9+
Microsoft.ML contains one other native library LDANative which is used by the LatentDirichletAllocationTransformer/LatentDirichletAllocationEstimator to support the LightLDA algorithm. If this component is used it will require the LightLDA native library. The native library is built my ML.NET for linux-arm, linux-arm64, linux-x64, osx-arm64 (M1), osx-x64, win-arm64, win-x64, win-x86. This library has only platform/CRT dependencies.
10+
11+
12+
13+
Some components that represent an algorithm or binding to another framework are factored into separate packages to allow opt-in to using those and their dependencies.
14+
15+
ML.NET redistributes Intel MKL as Microsoft.ML.MKL.Redist in which a minimized library is linked with just exports used. This component is follows the support matrix of Intel MKL and is only supported on x86 and x64 architectures: linux-x64, osx-x64 (no longer supported by Intel), win-x64, and win-x86. Similarly some components have light-up to use an Intel OneDAL implementation which is only supported on x64.
16+
17+
| NuGet Package | Entry-Point Components | Native Dependencies | Status | Notes |
18+
19+
|-------------------------------------|----------------------------------------------------------------|------------------------------------------------|----------|--------------------------------------------------------------------------------------------|
20+
21+
| `Microsoft.ML` | `MLContext`, core transforms, trainers | None | Stable | |
22+
23+
| `Microsoft.Extensions.ML` | `PredictionEnginePool` | None | Stable | |
24+
25+
| `Microsoft.ML.AutoML` | `AutoCatalog` for AutoML | \*As required by other components\* | Preview | Support varies based on components used |
26+
27+
| `Microsoft.ML.CodeGenerator` | | None | Preview | Part of AutoML |
28+
29+
| `Microsoft.ML.CpuMath` | | Optional native | Stable | Internal implementation; only used on .NET Framework |
30+
31+
| `Microsoft.ML.DataView` | `IDataView` | None | Stable | |
32+
33+
| `Microsoft.ML.DnnImageFeaturizer.\*` | | None | Preview | Data-only |
34+
35+
| `Microsoft.ML.Ensemble` | | None | Preview | Supports ML.NET component catalog |
36+
37+
| `Microsoft.ML.EntryPoints` | | None | Preview | Supports ML.NET component catalog |
38+
39+
| `Microsoft.ML.Experimental` | | None | Preview | Experimental API |
40+
41+
| `Microsoft.ML.FairLearn` | `FairlearnCatalog` | None | Preview | |
42+
43+
| `Microsoft.ML.FastTree` | `FastTreeRankingTrainer` | Optional native acceleration | Stable | Native library used on x86/x64; managed fallback |
44+
45+
| `Microsoft.ML.ImageAnalytics` | `MLImage` (image exchange type) | `libSkiaSharp` | Stable | Wrapper over SkiaSharp / Google Skia; supported where dependency is supported |
46+
47+
| `Microsoft.ML.LightGBM` | `LightGbm\*Trainer` | `LightGBM` | Stable | Wrapper over LightGBM; supported where dependency is supported |
48+
49+
| `Microsoft.ML.MKL.Components` | `SymbolicSgdLogisticRegressionBinaryTrainer` | Intel MKL | Stable | Only works where Intel MKL works |
50+
51+
| `Microsoft.ML.MKL.Redist` | Internal native Intel MKL | `libomp` | Stable | Not for direct reference; win-x86/x64 only |
52+
53+
| `Microsoft.ML.OneDal` | Internal native Intel OneDal | Intel OneDAL | Preview | Not for direct reference; x64 only |
54+
55+
| `Microsoft.ML.OnnxConverter` | Adds ONNX export support | `Microsoft.ML.OnnxRuntime` | Stable | Wrapper over ONNX Runtime; supports "bring your own" runtime |
56+
57+
| `Microsoft.ML.OnnxTransformer` | `OnnxCatalog` | `Microsoft.ML.OnnxRuntime` | Stable | Wrapper over ONNX Runtime; supports "bring your own" runtime |
58+
59+
| `Microsoft.ML.Parquet` | `ParquetLoader` | None | Preview | Uses managed Parquet.Net (port of Apache Parquet) |
60+
61+
| `Microsoft.ML.Recommender` | `MatrixFactorizationTrainer` | LIBMF (bundled) | Stable | Includes libmf built for all runtimes supported by ML.NET |
62+
63+
| `Microsoft.ML.TensorFlow` | `TensorFlowModel`, `Transformer`, `Estimator` | TensorFlow via `TensorFlow.NET` | Stable | Wrapper over TensorFlow; supports "bring your own" runtime |
64+
65+
| `Microsoft.ML.TimeSeries` | `ForecastingCatalog` | Intel MKL, `libomp` | Stable | Only works where Intel MKL works |
66+
67+
| `Microsoft.ML.TorchSharp` | `QATrainer`, `TextClassificationTrainer`, `SentenceSimilarityTrainer` | libTorch via `TorchSharp` | Preview | Wrapper over libTorch; supported where TorchSharp is supported |
68+
69+
| `Microsoft.ML.Vision` | `ImageClassificationTrainer` | TensorFlow | Stable | Depends on `Microsoft.ML.TensorFlow` for implementation |
70+
71+
72+
73+
74+
75+
Other packages:
76+
| NuGet Package | Entry-Point Components | Native Dependencies | Status | Notes |
77+
78+
|----------------------------------|--------------------------------------------------|-----------------------------------------------|------------|---------|
79+
80+
| `Microsoft.Data.Analysis` | `DataFrame` | `Apache.Arrow` | Preview | |
81+
82+
| `Microsoft.ML.GenAI.\*` | | | Preview | |
83+
84+
| `Microsoft.ML.Tokenizers.\*` | `Tokenizer` | | Stable | |
85+
86+
| `Microsoft.ML.SampleUtils` | | | Preview | |
87+
88+
89+
90+
91+

0 commit comments

Comments
 (0)