Add OnnxTransform for scoring Onnx 1.2 models - integrates Microsoft.ML.Scoring/Sonoma Library by jignparm · Pull Request #942 · dotnet/machinelearning

jignparm · 2018-09-19T00:00:47Z

Fixes issue #695
Fixes issue #892

This adds a new transform for scoring Onnx v1.2 models, leveraging an updated version of the scoring library at the link below.

https://www.nuget.org/packages/Microsoft.ML.Scoring/

dnfclas · 2018-09-19T00:00:58Z

All CLA requirements met. #Closed

Ivanidzo4ka · 2018-09-21T19:55:50Z

+
+  <ItemGroup>
+    <ProjectReference Include="../Microsoft.ML/Microsoft.ML.nupkgproj" />
+    <PackageReference Include="Microsoft.ML.Scoring" Version="1.0.4-dev47509" />


1.0.4-dev47509 [](start = 62, length = 14)

please put it to https://github.com/dotnet/machinelearning/blob/master/build/Dependencies.props #Closed

Ivanidzo4ka · 2018-09-21T20:03:36Z

+        internal const string LoaderSignature = "OnnxTransform";
+
+        public readonly string[] Inputs;
+        public readonly string[] Outputs;


Do you plan to support multiple input /outputs or only one?
It looks weird, you constantly cast single objects to arrays and back. #Closed

It's single input/output transform. I cleaned all the casts.

In reply to: 219612560 [](ancestors = 219612560)

Ivanidzo4ka · 2018-09-21T20:05:35Z

+{
+    public sealed class OnnxTransform : ITransformer, ICanSaveModel
+    {
+        public sealed class Arguments : TransformInputBase


Arguments [](start = 28, length = 9)

So far we have tendency to use Arguments class for entry points and command line, and to have separate class ColumnInfo with same functionality, but without all attributes.
And in our constructors we tend to use ColumnInfo class (except constructor for SignatureLoadDataTransform) #Pending

I was basing this on TensorFlowTransform which only has Arguments class. I looked at couple other transforms (MutualInformationFeatureSelectionTransform, GroupTransform) and they have Arguments only. Could you please point me to a sample transform that follows above? #Resolved

[Adding Shahab's response]
I was basing this on TensorFlowTransform which only has Arguments class. I looked at couple other transforms (MutualInformationFeatureSelectionTransform, GroupTransform) and they have Arguments only. Could you please point me to a sample transform that follows above?

In reply to: 219613034 [](ancestors = 219613034)

Ivanidzo4ka · 2018-09-21T20:05:46Z

+        }
+
+        // Factory method for SignatureLoadDataTransform
+        public static IDataTransform Create(IHostEnvironment env, ModelLoadContext ctx, IDataView input)


public [](start = 8, length = 6)

can make private #Closed

same for all factory methods

In reply to: 219613072 [](ancestors = 219613072)

Aren't some of the Create(...) need to be public, b/c they get called from another assembly? #Closed

Some, yes, probably one which returns IDataTransform. All other methods, just for our dependency framework, and can be private.

In reply to: 220260318 [](ancestors = 220260318)

Made them private

In reply to: 219613072 [](ancestors = 219613072)

Ok, thanks Ivan. We've made them private for now -- all unit tests are passing so far.

In reply to: 220309543 [](ancestors = 220309543,220260318)

Ivanidzo4ka · 2018-09-21T20:06:29Z

+            ctx.SaveNonEmptyString(_args.OutputColumn);
+        }
+
+        internal sealed class Mapper : IRowMapper


internal [](start = 8, length = 8)

why not private? #Closed

Made private #Closed

made private

In reply to: 219613222 [](ancestors = 219613222)

Ivanidzo4ka · 2018-09-21T20:09:22Z

+        /// After adaptation, you'd call GetTensor() on the IdvToTensorAdapter object to get the Tensor equivalent of
+        /// each row.
+        /// </summary>
+        internal sealed class IdvToTensorAdapter


IdvToTensorAdapter [](start = 30, length = 18)

I would move it to OnnxUtils #Closed

Moved to OnnxUtils #Closed

Moved to OnnxUtils.

In reply to: 219613978 [](ancestors = 219613978)

abgoswam · 2018-09-24T16:42:34Z

+            [Argument(ArgumentType.Required, HelpText = "Path to the onnx model file.", ShortName = "model", SortOrder = 0)]
+            public string ModelFile;
+
+            [Argument(ArgumentType.Multiple | ArgumentType.Required, HelpText = "TBD", SortOrder = 1)]


TBD [](start = 81, length = 3)

nit: needs to be updated #Resolved

Fixed #Resolved

Done!

In reply to: 219906640 [](ancestors = 219906640)

abgoswam · 2018-09-24T16:59:14Z

+
+        public static OnnxModel CreateFromBytes(byte[] modelBytes)
+        {
+            var tempModelDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString());


tempModelDir [](start = 16, length = 12)

when does this tempModelDir get cleaned up ?

abgoswam · 2018-09-24T17:03:38Z

+        public static OnnxModel CreateFromBytes(byte[] modelBytes)
+        {
+            var tempModelDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString());
+            Directory.CreateDirectory(tempModelDir);


tempModelDir [](start = 38, length = 12)

might consider ACL this temp dir .

Temp directories with models are considered executable code and considered security threat by .NET security team

abgoswam · 2018-09-24T17:25:53Z

+        </para>
+
+        <para>
+          This transform requires the <a href="https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.ML.TensorFlow/0.5.0-preview-26830-5">Microsoft.ML.TensorFlow</a> nuget to be installed.


Microsoft.ML.TensorFlow [](start = 150, length = 23)

is this a requirement for the OnnxTransform ? #Resolved

Jignesh, this should be Sonoma nuget, right? Could you please update it. #Resolved

The entire doc.xml has been modified. No reference to TF anywhere now.

In reply to: 219920734 [](ancestors = 219920734)

Yes, this doc has been rewritten. The version being commented on was obselete.

In reply to: 220279203 [](ancestors = 220279203)

abgoswam · 2018-09-24T17:28:39Z

+            {
+                _host.CheckNonWhiteSpace(args.ModelFile, nameof(args.ModelFile));
+                _host.CheckUserArg(File.Exists(args.ModelFile), nameof(args.ModelFile));
+                _model = new OnnxModel(args.ModelFile);


_model [](start = 16, length = 6)

is there a memory limitation of keeping everything in memory ? i would presume some Onnx models to be quite large
#Resolved

This API is what is supported in Sonoma currently. There's no alternative at this point. This can be a design discussion for the Lotus# library. Making as resolved.

In reply to: 219921638 [](ancestors = 219921638)

…machinelearning into jignparm/onnxtransform

Ivanidzo4ka · 2018-09-24T19:55:09Z

+        }
+
+        [Fact]
+        public void OnnxStatic()


Not related to this one, but I would love to have test which would ran ML.Net-> Onnx conversion and then score that model, and compare results. #WontFix

Yup, that's a great Uber-test to have. We can probably track that as a separate feature or task (since it's using 2 diff transforms). Let me know if we can close this for now.

In reply to: 219967750 [](ancestors = 219967750)

Ivanidzo4ka · 2018-09-24T20:24:35Z

+
+  <PropertyGroup>
+    <TargetFramework>netstandard2.0</TargetFramework>
+    <IncludeInPackage>Microsoft.ML.TensorFlow</IncludeInPackage>


Microsoft.ML.TensorFlow [](start = 22, length = 23)

Microsoft.ML.OnnxTransform #Closed

Ivanidzo4ka · 2018-09-26T00:47:27Z

+            try
+            {
+                pipe.Fit(invalidDataWrongVectorSize);
+                //Assert.False(true);


//Assert.False(true); [](start = 16, length = 21)

why this is commented? #Closed

Uncommented now.

In reply to: 220398186 [](ancestors = 220398186)

Ivanidzo4ka · 2018-09-26T00:47:40Z

+            var invalidDataWrongTypes = ComponentCreation.CreateDataView(Env, stringData);
+            var invalidDataWrongVectorSize = ComponentCreation.CreateDataView(Env, sizeData);
+            TestEstimatorCore(pipe, dataView, invalidInput: invalidDataWrongNames);
+            //TestEstimatorCore(pipe, dataView, invalidInput: invalidDataWrongTypes);


//TestEstimatorCore(pipe, dataView, invalidInput: invalidDataWrongTypes); [](start = 12, length = 73)

? #Closed

Uncommented

In reply to: 220398217 [](ancestors = 220398217)

Ivanidzo4ka · 2018-09-26T00:51:33Z

+        {
+            using (var env = new ConsoleEnvironment())
+            {
+                Assert.Equal(Maml.Main(new[] { @"showschema loader=Text{col=a:R4:0-3 col=b:R4:0-3} xf=OnnxTransform{inputs=a inputs=b outputs=c model={model_matmul/frozen_saved_model.pb}} in=f:\2.txt" }), (int)0);


inputs=a inputs=b outputs=c [](start = 116, length = 27)

in your argument class you have this:
public string InputColumn;
public string OutputColumn

you accept only one column as input and one as output.
also names are different. #Closed

Yes. Please see upated.

In reply to: 220398640 [](ancestors = 220398640)

Ivanidzo4ka · 2018-09-26T00:52:24Z

    <ProjectReference Include="..\..\src\Microsoft.ML.StandardLearners\Microsoft.ML.StandardLearners.csproj" />
    <ProjectReference Include="..\..\src\Microsoft.ML.Onnx\Microsoft.ML.Onnx.csproj" />
    <ProjectReference Include="..\..\src\Microsoft.ML.TensorFlow\Microsoft.ML.TensorFlow.csproj" />
+    <ProjectReference Include="..\..\src\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.csproj" />


revert whole file. #Pending

good catch. Done

In reply to: 220398733 [](ancestors = 220398733)

You still have package reference on Onnx.TestModels.

In reply to: 220411170 [](ancestors = 220411170,220398733)

Ivanidzo4ka · 2018-09-26T01:04:02Z

+
+            resultDic[Transformer.Output] = new SchemaShape.Column(Transformer.Output,
+                Transformer.OutputType.IsKnownSizeVector ? SchemaShape.Column.VectorKind.Vector
+                : SchemaShape.Column.VectorKind.VariableVector, Transformer.OutputType.ItemType, false);


Transformer.OutputType.ItemType [](start = 64, length = 31)

Either modify OnnxUtils.CopyTo to support other types rather than float, or put NumberType.R4 here. #Closed

Done.

In reply to: 220400133 [](ancestors = 220400133)

Ivanidzo4ka · 2018-09-26T01:04:48Z

+                _host.AssertValue(input);
+                _host.Assert(typeof(T) == _outputItemRawType);
+
+                ValueGetter<VBuffer<T>> valuegetter = (ref VBuffer<T> dst) =>


valuegetter [](start = 40, length = 11)

camelCase please #Closed

Done.

In reply to: 220400214 [](ancestors = 220400214)

Ivanidzo4ka · 2018-09-26T01:08:54Z

+  <PropertyGroup>
+    <TargetFramework>netstandard2.0</TargetFramework>
+    <IncludeInPackage>Microsoft.ML.OnnxTransform</IncludeInPackage>
+    <DefineConstants>CORECLR</DefineConstants>


This one is unnecessary #Closed

removed

In reply to: 220400768 [](ancestors = 220400768)

Ivanidzo4ka · 2018-09-26T01:09:34Z

+            <description>The name of each output column should match one of the operations in the Tensorflow graph.</description>
+          </item>
+          <item>
+            <description>Currently, float and double are the only acceptable data types for input/output.</description>


double [](start = 46, length = 6)

You throw exception in case of double type as output column. #Closed

Fixed this to only 'float'.

In reply to: 220400879 [](ancestors = 220400879)

Ivanidzo4ka · 2018-09-26T01:10:13Z

+        </list>
+
+        The inputs and outputs of a TensorFlow model can be obtained using the <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms/README.md#inspecting-graphs">
+          <code>summarize_graph</code> tool


Should be onnx specific or removed. #Closed

replaced with netron tool.

In reply to: 220400976 [](ancestors = 220400976)

Ivanidzo4ka · 2018-09-26T17:06:49Z

@@ -0,0 +1,404 @@
+using Microsoft.ML.Runtime;


nit: need header

Ivanidzo4ka · 2018-09-26T17:09:06Z

  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.ML.TensorFlow.TestModels" Version="0.0.3-test" />
+    <PackageReference Include="Microsoft.ML.Onnx.TestModels" Version="0.0.2-test" />


[](start = 4, length = 80)

You don't need this line anymore.

Ivanidzo4ka

helloguo · 2018-10-24T21:43:01Z

Is Microsoft.ML.Scoring going to be a part of ML.NET and open source?

initial checkin

d67eb57

jignparm requested a review from zeahmed September 19, 2018 00:01

jignparm changed the title ~~initial checkin~~ PR - OnnxTransform Sep 19, 2018

added 1 file

1402d19

jignparm changed the title ~~PR - OnnxTransform~~ [WIP] PR - OnnxTransform Sep 19, 2018

jignparm changed the title ~~[WIP] PR - OnnxTransform~~ [WIP] Add OnnxTransform for scoring Onnx 1.2 models - integrates Microsoft.ML.Scoring/Sonoma Library Sep 19, 2018

jignparm added 3 commits September 19, 2018 01:53

Added nugets and project so compile successfully

d7aefce

Added unit tests -- need further debugging

b5482df

Merge branch 'master' into jignparm/onnxtransform

40f857e

Ivanidzo4ka reviewed Sep 21, 2018

View reviewed changes

Comment thread src/Microsoft.ML.OnnxTransform/Microsoft.ML.OnnxTransform.csproj

Ivanidzo4ka reviewed Sep 21, 2018

View reviewed changes

Fixed schema propagation tests

46d7821

abgoswam reviewed Sep 24, 2018

View reviewed changes

Comment thread src/Microsoft.ML.OnnxTransform/OnnxTransform.cs

abgoswam reviewed Sep 24, 2018

View reviewed changes

Comment thread src/Microsoft.ML.OnnxTransform/OnnxTransform.cs

abgoswam reviewed Sep 24, 2018

View reviewed changes

Merge remote-tracking branch 'upstream/master' into jignesh/onnx

4bf00de

abgoswam reviewed Sep 24, 2018

View reviewed changes

Comment thread src/Microsoft.ML.OnnxTransform/OnnxUtils.cs

abgoswam reviewed Sep 24, 2018

View reviewed changes

Merge branch 'jignparm/onnxtransform' of https://github.com/jignparm/…

dd956be

…machinelearning into jignparm/onnxtransform

Ivanidzo4ka reviewed Sep 24, 2018

View reviewed changes

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

added PR comments

d554822

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

Comment thread test/Microsoft.ML.OnnxTransformTest/OnnxTransformTests.cs Outdated

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

Comment thread test/Microsoft.ML.OnnxTransformTest/OnnxTransformTests.cs Outdated

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

Comment thread test/Microsoft.ML.OnnxTransformTest/OnnxTransformTests.cs

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

Comment thread src/Microsoft.ML.OnnxTransform/OnnxTransform.cs Outdated

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

jignparm added 4 commits September 26, 2018 01:33

Fixed PR code reviews

3069de6

more PR comments

ca61e26

Addressed Ivans PR comments

d95e323

added back onnx proj, removed onnxtransform proj

24d34b4

jignparm changed the title ~~[WIP] Add OnnxTransform for scoring Onnx 1.2 models - integrates Microsoft.ML.Scoring/Sonoma Library~~ Add OnnxTransform for scoring Onnx 1.2 models - integrates Microsoft.ML.Scoring/Sonoma Library Sep 26, 2018

jignparm added 5 commits September 26, 2018 07:45

small edit to kick off build

0182ca1

doc changes

cc14268

fixed gt to gt;

aa17a35

small change to kick off build

4c7821f

create IDV from array instead of list of array

316a5a1

Ivanidzo4ka reviewed Sep 26, 2018

View reviewed changes

Ivanidzo4ka approved these changes Sep 26, 2018

View reviewed changes

shauheen merged commit f6d850f into dotnet:master Sep 26, 2018

ghost locked as resolved and limited conversation to collaborators Mar 28, 2022

Conversation

jignparm commented Sep 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dnfclas commented Sep 19, 2018 • edited by jignparm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited by jignparm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmoradims Sep 25, 2018 • edited by jignparm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmoradims Sep 25, 2018 • edited by Ivanidzo4ka Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmoradims Sep 25, 2018 • edited by Ivanidzo4ka Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ivanidzo4ka Sep 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmoradims Sep 25, 2018 • edited by Ivanidzo4ka Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

abgoswam Sep 24, 2018 • edited by jignparm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shmoradims Sep 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jignparm commented Sep 19, 2018 •

edited

Loading

dnfclas commented Sep 19, 2018 •

edited by jignparm

Loading

Ivanidzo4ka Sep 21, 2018 •

edited

Loading

Ivanidzo4ka Sep 21, 2018 •

edited

Loading

Ivanidzo4ka Sep 21, 2018 •

edited by jignparm

Loading

shmoradims Sep 25, 2018 •

edited by jignparm

Loading

Ivanidzo4ka Sep 21, 2018 •

edited

Loading

shmoradims Sep 25, 2018 •

edited by Ivanidzo4ka

Loading

Ivanidzo4ka Sep 21, 2018 •

edited

Loading

shmoradims Sep 25, 2018 •

edited by Ivanidzo4ka

Loading

Ivanidzo4ka Sep 21, 2018 •

edited

Loading

shmoradims Sep 25, 2018 •

edited by Ivanidzo4ka

Loading

abgoswam Sep 24, 2018 •

edited by jignparm

Loading

shmoradims Sep 25, 2018 •

edited

Loading

abgoswam Sep 24, 2018 •

edited by jignparm

Loading

shmoradims Sep 25, 2018 •

edited by jignparm

Loading

abgoswam Sep 24, 2018 •

edited by jignparm

Loading

Ivanidzo4ka Sep 24, 2018 •

edited by jignparm

Loading

Ivanidzo4ka Sep 24, 2018 •

edited

Loading

Ivanidzo4ka Sep 26, 2018 •

edited

Loading

Ivanidzo4ka Sep 26, 2018 •

edited

Loading

Ivanidzo4ka Sep 26, 2018 •

edited

Loading