New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tensorflow GetModelSchema should support unfrozen models #2112

Merged
merged 3 commits into from Jan 11, 2019

Conversation

Projects
None yet
5 participants
@Ivanidzo4ka
Copy link
Member

Ivanidzo4ka commented Jan 10, 2019

Fix #2102

@Ivanidzo4ka Ivanidzo4ka requested review from abgoswam and yaeldekel Jan 10, 2019

var mlContext = new MLContext(seed: 1, conc: 1);
var loadModelSchema = TensorFlowUtils.GetModelSchema(mlContext, modelLocation);
Assert.Equal(335, loadModelSchema.Count);

This comment has been minimized.

@abgoswam

abgoswam Jan 10, 2019

Member

Assert.Equal(335, loadModelSchema.Count [](start = 12, length = 39)

am curious , what does this count signify ? #Closed

This comment has been minimized.

@Ivanidzo4ka

Ivanidzo4ka Jan 10, 2019

Member

It's all the nodes inside tensor flow graph.
I just wanted to test GetModelSchema on unfrozen model.


In reply to: 246930752 [](ancestors = 246930752)

/// <param name="modelFile">The name of the file containing the TensorFlow model. Currently only frozen model
/// format is supported.</param>
public static Schema GetModelSchema(IExceptionContext ectx, string modelFile)
public static Schema GetModelSchema(IHostEnvironment env, string modelFile)

This comment has been minimized.

@abgoswam

abgoswam Jan 10, 2019

Member

modelFile [](start = 73, length = 9)

for consistency, can we call this modelPath #Closed

}

/// <summary>
/// This is a convenience method for iterating over the nodes of a TensorFlow model graph. It
/// iterates over the columns of the <see cref="ISchema"/> returned by <see cref="GetModelSchema(IExceptionContext, string)"/>,
/// iterates over the columns of the <see cref="ISchema"/> returned by <see cref="GetModelSchema(IHostEnvironment, string)"/>,
/// and for each one it returns a tuple containing the name, operation type, column type and an array of input node names.
/// This method is convenient for filtering nodes based on certain criteria, for example, by the operation type.
/// </summary>
/// <param name="modelFile"></param>
/// <returns></returns>
public static IEnumerable<(string, string, ColumnType, string[])> GetModelNodes(string modelFile)

This comment has been minimized.

@abgoswam

abgoswam Jan 10, 2019

Member

modelFile [](start = 95, length = 9)

for consistency, can we call this modelPath #Closed

}

/// <summary>
/// This is a convenience method for iterating over the nodes of a TensorFlow model graph. It
/// iterates over the columns of the <see cref="ISchema"/> returned by <see cref="GetModelSchema(IExceptionContext, string)"/>,
/// iterates over the columns of the <see cref="ISchema"/> returned by <see cref="GetModelSchema(IHostEnvironment, string)"/>,
/// and for each one it returns a tuple containing the name, operation type, column type and an array of input node names.
/// This method is convenient for filtering nodes based on certain criteria, for example, by the operation type.
/// </summary>
/// <param name="modelFile"></param>

This comment has been minimized.

@abgoswam

abgoswam Jan 10, 2019

Member

/// [](start = 8, length = 36)

some description ? #Closed

@@ -75,27 +75,26 @@ internal static Schema GetModelSchema(IExceptionContext ectx, TFGraph graph, str
/// of kind <see cref="OpType"/>, indicating the operation type of the node, and if that node has inputs in the graph,
/// it contains metadata of kind <see cref="InputOps"/>, indicating the names of the input nodes.
/// </summary>
/// <param name="ectx">An <see cref="IExceptionContext"/>.</param>
/// <param name="env">The environment to use.</param>
/// <param name="modelFile">The name of the file containing the TensorFlow model. Currently only frozen model

This comment has been minimized.

@abgoswam

abgoswam Jan 10, 2019

Member

Currently only frozen model [](start = 90, length = 27)

so this would need to be updated i presume #Closed

@Ivanidzo4ka Ivanidzo4ka requested a review from sfilipi Jan 10, 2019

@singlis
Copy link
Member

singlis left a comment

:shipit:

@abgoswam
Copy link
Member

abgoswam left a comment

:shipit:

@Ivanidzo4ka Ivanidzo4ka merged commit 72a4eb6 into dotnet:master Jan 11, 2019

2 checks passed

MachineLearning-CI #20190110.40 succeeded
Details
license/cla All CLA requirements met.
Details

wschin added a commit to wschin/machinelearning that referenced this pull request Jan 11, 2019

{
var schema = GetModelSchema(null, modelFile);
var schema = GetModelSchema(new MLContext(), modelPath);

This comment has been minimized.

@eerhardt

eerhardt Jan 11, 2019

Member

This is not correct. If you need an IHostEnvironment, you should be taking one as a parameter. It is wrong to create new environment (MLContext) out of thin air.

/cc @TomFinley

This comment has been minimized.

@yaeldekel

yaeldekel Jan 14, 2019

Member

This is only used as an IExceptionContext, so if we remove the check that env!=null on line 325, we can pass null.


In reply to: 247276711 [](ancestors = 247276711)

@abgoswam

This comment has been minimized.

Copy link
Member

abgoswam commented Jan 13, 2019

@Ivanidzo4ka . I am seeing sporadic failures in the TF tests .

On the local machine I get a pop-up saying .NET Core Host stopped working.
On the build machine I see message saying the active test run was aborted

If I skip the TF tests, the build goes through fine. Could this be related to this change ?

Note: the failures are sporadic.

public static Schema GetModelSchema(IExceptionContext ectx, string modelFile)
/// <param name="env">The environment to use.</param>
/// <param name="modelPath">Model to load.</param>
public static Schema GetModelSchema(IHostEnvironment env, string modelPath)

This comment has been minimized.

@yaeldekel

yaeldekel Jan 14, 2019

Member

env [](start = 61, length = 3)

This is only used as an IExceptionContext, so instead of changing it here to IHostEnvironment, you can change LoadTensorFlowModel and GetSession to take an IExceptionContext.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment