Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removed ability to save filters from legacy filter code #5338

Merged
merged 3 commits into from
Aug 10, 2020
Merged

Removed ability to save filters from legacy filter code #5338

merged 3 commits into from
Aug 10, 2020

Conversation

harishsk
Copy link
Contributor

@harishsk harishsk commented Aug 6, 2020

Fixes #5335

The offending code in ResultProcessor wasn't being used at all. So I have deleted that.

And as per @yaeldekel the StatefulFilterTransform is a legacy filter from TLC and is not part of the ML.NET public API. It is being used internally in some time series code. But it doesn't need the ability to be saved.

We should at some point, upgrade the time series code to use the new CustomMappingTransform and delete the StatefulFilterTransform.

Until then I have deleted the SerializableLambdaTransform and the associated functionality in LambdaTransformBase.

@harishsk harishsk requested review from yaeldekel, a team and antoniovs1029 August 6, 2020 00:25
@@ -219,32 +174,12 @@ private static ITransformer Create(IHostEnvironment env, ModelLoadContext ctx)
/// </summary>
internal abstract class LambdaTransformBase : ICanSaveModel
Copy link

@yaeldekel yaeldekel Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ICanSaveModel [](start = 50, length = 13)

Since this doesn't need to implement ICanSaveModel, and there is no other functionality in this base class, I would delete it. You mentioned that it is being used internally only as an IDataView, in that case I would make the deriving classes (I only found one, StatefulFilterTransform) implement IDataView instead of ITransformTemplate. #Resolved

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yaeldekel. I have deleted LambdaTransformBase. I replaced ITransformTemplate with IDataTransform instead of IDataView because SequentialTransformBase in the time series code was relying on it being IDataTransform


In reply to: 466166529 [](ancestors = 466166529)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind. Your advice was correct. The earlier fix resulted in test failures. I have now changed StatefulFilterTransform to implement IDataView


In reply to: 466589874 [](ancestors = 466589874,466166529)

Copy link
Member

@antoniovs1029 antoniovs1029 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@antoniovs1029
Copy link
Member

Just to mention that this also is meant to fix CA2301 on #5334 😄

@codecov
Copy link

codecov bot commented Aug 6, 2020

Codecov Report

Merging #5338 into master will decrease coverage by 4.07%.
The diff coverage is 73.33%.

@@            Coverage Diff             @@
##           master    #5338      +/-   ##
==========================================
- Coverage   73.92%   69.84%   -4.08%     
==========================================
  Files        1020      775     -245     
  Lines      190126   145190   -44936     
  Branches    20437    18529    -1908     
==========================================
- Hits       140549   101414   -39135     
+ Misses      44048    38611    -5437     
+ Partials     5529     5165     -364     
Flag Coverage Δ
#Debug 69.84% <73.33%> (-4.08%) ⬇️
#production 69.84% <73.33%> (+0.16%) ⬆️
#test ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...rc/Microsoft.ML.ResultProcessor/ResultProcessor.cs 46.85% <0.00%> (+2.19%) ⬆️
...Microsoft.ML.TimeSeries/SequentialTransformBase.cs 77.01% <ø> (ø)
...crosoft.ML.TimeSeries/SequentialTransformerBase.cs 55.35% <ø> (ø)
...Microsoft.ML.Transforms/StatefulFilterTransform.cs 83.03% <75.00%> (-9.07%) ⬇️
src/Microsoft.ML.Transforms/LambdaTransform.cs 94.33% <100.00%> (+27.37%) ⬆️
src/Microsoft.ML.Maml/MAML.cs 23.78% <0.00%> (-0.98%) ⬇️
...StandardTrainers/Standard/LinearModelParameters.cs 66.32% <0.00%> (-0.26%) ⬇️
...soft.ML.Predictor.Tests/CmdLine/CmdIndenterTest.cs
test/Microsoft.ML.Benchmarks/Harness/Metrics.cs
... and 253 more

@harishsk harishsk merged commit 4bc2d78 into dotnet:master Aug 10, 2020
@antoniovs1029 antoniovs1029 mentioned this pull request Aug 10, 2020
12 tasks
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SerializationBinder must be set for BinaryFormatter
3 participants