New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorFlowTransform example in Microsoft.ML.Samples #1812

Merged
merged 4 commits into from Dec 6, 2018

Conversation

Projects
None yet
4 participants
@abgoswam
Member

abgoswam commented Dec 4, 2018

Fixes #1715

  • Adding example for TensorFlowTransform in Microsoft.ML.Samples.
<ProjectReference Include="..\..\..\src\Microsoft.ML.TimeSeries\Microsoft.ML.TimeSeries.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.ML.OnnxTransform\Microsoft.ML.OnnxTransform.csproj" />
<NativeAssemblyReference Include="CpuMathNative" />
<NativeAssemblyReference Include="FastTreeNative" />
<NativeAssemblyReference Include="MatrixFactorizationNative" />
<NativeAssemblyReference Include="LdaNative" />
<PackageReference Include="Microsoft.ML.TensorFlow.Redist" Version="0.7.0" />

This comment has been minimized.

@Ivanidzo4ka

Ivanidzo4ka Dec 4, 2018

Member

[](start = 4, length = 77)

Is this really necessary?
I've look in our test projects, and we don't reference it.

This comment has been minimized.

@abgoswam

abgoswam Dec 4, 2018

Member

This was a bit surprising for me too. Without adding the Microsoft.ML.TensorFlow.Redist nuget, the Samples project complains saying it does not find tensorflow.dll

However, as you pointed out, the test project does not reference the Microsoft.ML.TensorFlow.Redist nuget.

@yaeldekel do you know how the test project manages to get the tensorflow.dll without explicitly referencing the Redist nuget ?


In reply to: 238764544 [](ancestors = 238764544)

This comment has been minimized.

@yaeldekel

yaeldekel Dec 6, 2018

Member

Discussed with Abhishek offline and figured out how to point to the tensorflow dll.


In reply to: 238786038 [](ancestors = 238786038,238764544)

This comment has been minimized.

@abgoswam

abgoswam Dec 6, 2018

Member

The test project has a NativeAssemblyReference which pulls in the dlls. Instead of doing that, we are using the TensorFlow.Redist package inside the Samples project.

@yaeldekel . I tried using the TensorFlow nuget instead of the Redist. However that leads to an issue - the TensorFlow nuget has dependency on the latest "official" ML.NET nuget (0.8). Code-changes which have gone in after the latest official release start complaining during build.

For now, I am keeping this as-is i.e. take nuget dependency on the Redist package + add project reference to the Microsoft.ML.TensorFlow.csproj


In reply to: 239576832 [](ancestors = 239576832,238786038,238764544)

var idv = mlContext.CreateStreamingDataView(data);
// Create a ML pipeline.
var pipeline = mlContext.Transforms.ScoreTensorFlowModel(modelLocation, new[] { "input" }, new[] { "output" });

This comment has been minimized.

@Ivanidzo4ka

Ivanidzo4ka Dec 4, 2018

Member

"output" [](start = 111, length = 8)

nameof(OutputScores.output)
same for input column #Resolved

//----------
}
private const int imageHeight = 224;

This comment has been minimized.

@montebhoover

montebhoover Dec 5, 2018

Member

Not sure how much it matters, I think the "normal" dimensions for the resnet model are 299x299. I only say that because that's required for the CNTK version and the tensorflow version has 299 in the filename, so that is not very scientific. #Resolved

This comment has been minimized.

@abgoswam

abgoswam Dec 6, 2018

Member

Looking up the author's github repo the input shape for ResNet 101 models is 224x224x3

In the CNTK based transform we had in the internal repo also takes input size 224x224 for ResNet 101 models.

paper


In reply to: 239240041 [](ancestors = 239240041)

This comment has been minimized.

@montebhoover

montebhoover Dec 6, 2018

Member

Ah, sorry about that then. Carry on. #Resolved

{
// Download the ResNet 101 model from the location below.
// https://storage.googleapis.com/download.tensorflow.org/models/tflite_11_05_08/resnet_v2_101.tgz
var modelLocation = @"resnet_v2_101/resnet_v2_101_299_frozen.pb";

This comment has been minimized.

@montebhoover

montebhoover Dec 5, 2018

Member

Maybe just preference, but I would put @"Dynamic/resnet_v2_101/resnet_v2_101_299_frozen.pb. I dropped the model into the same directory as the source file (and set "copy to output"), but that doesn't work by default because visual studio moves the binaries outside Dynamic in the bin. Adding "Dynamic" to the path makes it work by default, but you can decide if that is helpful or not here. #Resolved

This comment has been minimized.

@abgoswam

abgoswam Dec 6, 2018

Member

For specifying path to model file, I am following same pattern as the OnnxTransform. (prefer to keep both consistent)


In reply to: 239241951 [](ancestors = 239241951)

This comment has been minimized.

@montebhoover

montebhoover Dec 6, 2018

Member

Ah, sounds good.


In reply to: 239551754 [](ancestors = 239551754,239241951)

@@ -6,7 +6,7 @@ internal static class Program
{
static void Main(string[] args)
{
LdaTransformExample.LdaTransform();

This comment has been minimized.

@yaeldekel

yaeldekel Dec 6, 2018

Member

LdaTransformExample [](start = 12, length = 19)

Why is this being replaced? Don't we need the LDA sample anymore? #Resolved

This comment has been minimized.

@abgoswam

abgoswam Dec 6, 2018

Member

This is just there to invoke the particular example we want to run. The LDA sample still exists


In reply to: 239576064 [](ancestors = 239576064)

@abgoswam abgoswam merged commit 33c1884 into dotnet:master Dec 6, 2018

2 checks passed

MachineLearning-CI #20181206.11 succeeded
Details
license/cla All CLA requirements met.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment