Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only for test - test add fail retry for all tests #4510

Open
wants to merge 14 commits into
base: master
from

Conversation

@frank-dong-ms
Copy link
Member

frank-dong-ms commented Nov 28, 2019

Add MLNETFactAttribute and replace default Fact Attribute:

  1. every test case will retry for 3 times at max by default if the test fails
  2. log flaky tests to a sql db to better estimate which tests are fail often
  3. add default timeout for every tests
  4. refine logging
@codecov

This comment has been minimized.

Copy link

codecov bot commented Dec 4, 2019

Codecov Report

❗️ No coverage uploaded for pull request base (master@9fc8f1c). Click here to learn what that means.
The diff coverage is 48.25%.

@@            Coverage Diff            @@
##             master    #4510   +/-   ##
=========================================
  Coverage          ?   75.08%           
=========================================
  Files             ?      913           
  Lines             ?   160330           
  Branches          ?    17263           
=========================================
  Hits              ?   120382           
  Misses            ?    35130           
  Partials          ?     4818
Flag Coverage Δ
#Debug 75.08% <48.25%> (?)
#production 70.51% <ø> (?)
#test 90.09% <48.25%> (?)
Impacted Files Coverage Δ
test/Microsoft.ML.Functional.Tests/DataIO.cs 100% <ø> (ø)
test/Microsoft.ML.Functional.Tests/Training.cs 100% <ø> (ø)
...osoft.ML.CodeAnalyzer.Tests/Code/BestFriendTest.cs 100% <ø> (ø)
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs 100% <ø> (ø)
...crosoft.ML.AutoML.Tests/TransformInferenceTests.cs 100% <ø> (ø)
...est/Microsoft.ML.Core.Tests/UnitTests/DataTypes.cs 99.35% <ø> (ø)
...est/Microsoft.ML.Core.Tests/UnitTests/TestHosts.cs 100% <ø> (ø)
...ios/IrisPlantClassificationWithStringLabelTests.cs 98.63% <ø> (ø)
...t/Microsoft.ML.Core.Tests/UnitTests/ColumnTypes.cs 75.3% <ø> (ø)
test/Microsoft.ML.TimeSeries.Tests/TimeSeries.cs 87.85% <ø> (ø)
... and 134 more
@frank-dong-ms frank-dong-ms marked this pull request as ready for review Dec 5, 2019
@frank-dong-ms frank-dong-ms requested review from dotnet/mlnet-automl as code owners Dec 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.