Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test stable cross validation iris #4625

Conversation

@frank-dong-ms
Copy link
Member

frank-dong-ms commented Jan 4, 2020

several fluky tests that call stack seems similar, add some debug log to help trouble shoot:

Microsoft.ML.Tests.Scenarios.Api.CookbookSamples.CookbookSamplesDynamicApi.CrossValidationIris [48s 317ms]
Error Message:
Assert failed: longIdx=32, invariants.Length=32
Expected: True
Actual: False
Stack Trace:
at Microsoft.ML.Internal.Internallearn.Test.GlobalBase.AssertHandler(String msg, IExceptionContext ectx) in F:\workspace_work\1\s\test\Microsoft.ML.TestFramework\GlobalBase.cs:line 71
at Microsoft.ML.Runtime.Contracts.DbgFailCore(String msg, IExceptionContext ctx) in F:\workspace_work\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 781
at Microsoft.ML.Runtime.Contracts.DbgFail(String msg) in F:\workspace_work\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 794
at Microsoft.ML.Runtime.Contracts.Assert(Boolean f, String msg) in F:\workspace_work\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 852
at Microsoft.ML.Trainers.SdcaTrainerBase3.TrainCore(IChannel ch, RoleMappedData data, LinearModelParameters predictor, Int32 weightSetCount) in F:\workspace\_work\1\s\src\Microsoft.ML.StandardTrainers\Standard\SdcaBinary.cs:line 574 at Microsoft.ML.Trainers.StochasticTrainerBase2.TrainModelCore(TrainContext context) in F:\workspace_work\1\s\src\Microsoft.ML.StandardTrainers\Standard\StochasticTrainerBase.cs:line 43
at Microsoft.ML.Trainers.TrainerEstimatorBase2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in F:\workspace\_work\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 157 at Microsoft.ML.Trainers.TrainerEstimatorBase2.Fit(IDataView input) in F:\workspace_work\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 77
at Microsoft.ML.Data.EstimatorChain1.Fit(IDataView input) in F:\workspace\_work\1\s\src\Microsoft.ML.Data\DataLoadSave\EstimatorChain.cs:line 67 at Microsoft.ML.TrainCatalogBase.CrossValidateTrain(IDataView data, IEstimator1 estimator, Int32 numFolds, String samplingKeyColumn, Nullable1 seed) in F:\workspace\_work\1\s\src\Microsoft.ML.Data\TrainCatalog.cs:line 104 at Microsoft.ML.MulticlassClassificationCatalog.CrossValidate(IDataView data, IEstimator1 estimator, Int32 numberOfFolds, String labelColumnName, String samplingKeyColumnName, Nullable`1 seed) in F:\workspace_work\1\s\src\Microsoft.ML.Data\TrainCatalog.cs:line 535
at Microsoft.ML.Tests.Scenarios.Api.CookbookSamples.CookbookSamplesDynamicApi.CrossValidationOn(String dataPath) in F:\workspace_work\1\s\test\Microsoft.ML.Tests\Scenarios\Api\CookbookSamples\CookbookSamplesDynamicApi.cs:line 590
at Microsoft.ML.Tests.Scenarios.Api.CookbookSamples.CookbookSamplesDynamicApi.CrossValidationIris() in F:\workspace_work\1\s\test\Microsoft.ML.Tests\Scenarios\Api\CookbookSamples\CookbookSamplesDynamicApi.cs:line 553

Microsoft.ML.RunTests.TestEntryPoints.TestOvaMacro [218ms]
Error Message:
Assert failed: longIdx=139, invariants.Length=139
Expected: True
Actual: False
Stack Trace:
at Microsoft.ML.Internal.Internallearn.Test.GlobalBase.AssertHandler(String msg, IExceptionContext ectx) in /__w/1/s/test/Microsoft.ML.TestFramework/GlobalBase.cs:line 71
at Microsoft.ML.Runtime.Contracts.DbgFailCore(String msg, IExceptionContext ctx) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 781
at Microsoft.ML.Runtime.Contracts.DbgFail(String msg) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 794
at Microsoft.ML.Runtime.Contracts.Assert(Boolean f, String msg) in /__w/1/s/src/Microsoft.ML.Core/Utilities/Contracts.cs:line 852
at Microsoft.ML.Trainers.SdcaTrainerBase3.TrainCore(IChannel ch, RoleMappedData data, LinearModelParameters predictor, Int32 weightSetCount) in /__w/1/s/src/Microsoft.ML.StandardTrainers/Standard/SdcaBinary.cs:line 574 at Microsoft.ML.Trainers.StochasticTrainerBase2.TrainModelCore(TrainContext context) in /__w/1/s/src/Microsoft.ML.StandardTrainers/Standard/StochasticTrainerBase.cs:line 43
at Microsoft.ML.Trainers.TrainerEstimatorBase2.Microsoft.ML.ITrainer<Microsoft.ML.IPredictor>.Train(TrainContext context) in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerEstimatorBase.cs:line 100 at Microsoft.ML.Trainers.TrainerEstimatorBase2.Microsoft.ML.ITrainer.Train(TrainContext context) in /__w/1/s/src/Microsoft.ML.Data/Training/TrainerEstimatorBase.cs:line 168
at Microsoft.ML.Data.TrainUtils.TrainCore(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, RoleMappedData validData, IComponentFactory1 calibrator, Int32 maxCalibrationExamples, Nullable1 cacheData, IPredictor inputPredictor, RoleMappedData testData) in /__w/1/s/src/Microsoft.ML.Data/Commands/TrainCommand.cs:line 280
at Microsoft.ML.Data.TrainUtils.Train(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, IComponentFactory1 calibrator, Int32 maxCalibrationExamples) in /__w/1/s/src/Microsoft.ML.Data/Commands/TrainCommand.cs:line 249 at Microsoft.ML.EntryPoints.TrainerEntryPointsUtils.Train[TArg,TOut](IHost host, TArg input, Func1 createTrainer, Func1 getLabel, Func1 getWeight, Func1 getGroup, Func1 getName, Func`1 getCustom, ICalibratorTrainerFactory calibrator, Int32 maxCalibrationExamples) in /__w/1/s/src/Microsoft.ML.Data/EntryPoints/InputBase.cs:line 119
at Microsoft.ML.Trainers.Sdca.TrainBinary(IHostEnvironment env, Options input) in /__w/1/s/src/Microsoft.ML.StandardTrainers/Standard/SdcaBinary.cs:line 2488
Standard Output Messages:
Test TestOvaMacro: aborted: passed

[xUnit.net 00:00:46.41] Microsoft.ML.RunTests.TestEntryPoints.EntryPointChainedCrossValMacros(iteration: 44) [FAIL]
77-th running...
X Microsoft.ML.RunTests.TestEntryPoints.EntryPointChainedCrossValMacros(iteration: 44) [173ms]
Error Message:
Assert failed: longIdx=335, invariants.Length=335
Expected: True
Actual: False
Stack Trace:
at Microsoft.ML.Internal.Internallearn.Test.GlobalBase.AssertHandler(String msg, IExceptionContext ectx) in D:\a\1\s\test\Microsoft.ML.TestFramework\GlobalBase.cs:line 71
at Microsoft.ML.Runtime.Contracts.DbgFailCore(String msg, IExceptionContext ctx) in D:\a\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 781
at Microsoft.ML.Runtime.Contracts.DbgFail(String msg) in D:\a\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 794
at Microsoft.ML.Runtime.Contracts.Assert(Boolean f, String msg) in D:\a\1\s\src\Microsoft.ML.Core\Utilities\Contracts.cs:line 852
at Microsoft.ML.Trainers.SdcaTrainerBase3.TrainCore(IChannel ch, RoleMappedData data, LinearModelParameters predictor, Int32 weightSetCount) in D:\a\1\s\src\Microsoft.ML.StandardTrainers\Standard\SdcaBinary.cs:line 574 at Microsoft.ML.Trainers.StochasticTrainerBase2.TrainModelCore(TrainContext context) in D:\a\1\s\src\Microsoft.ML.StandardTrainers\Standard\StochasticTrainerBase.cs:line 43
at Microsoft.ML.Trainers.TrainerEstimatorBase2.Microsoft.ML.ITrainer<Microsoft.ML.IPredictor>.Train(TrainContext context) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 100 at Microsoft.ML.Trainers.TrainerEstimatorBase2.Microsoft.ML.ITrainer.Train(TrainContext context) in D:\a\1\s\src\Microsoft.ML.Data\Training\TrainerEstimatorBase.cs:line 168
at Microsoft.ML.Data.TrainUtils.TrainCore(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, RoleMappedData validData, IComponentFactory1 calibrator, Int32 maxCalibrationExamples, Nullable1 cacheData, IPredictor inputPredictor, RoleMappedData testData) in D:\a\1\s\src\Microsoft.ML.Data\Commands\TrainCommand.cs:line 280
at Microsoft.ML.Data.TrainUtils.Train(IHostEnvironment env, IChannel ch, RoleMappedData data, ITrainer trainer, IComponentFactory1 calibrator, Int32 maxCalibrationExamples) in D:\a\1\s\src\Microsoft.ML.Data\Commands\TrainCommand.cs:line 249 at Microsoft.ML.EntryPoints.TrainerEntryPointsUtils.Train[TArg,TOut](IHost host, TArg input, Func1 createTrainer, Func1 getLabel, Func1 getWeight, Func1 getGroup, Func1 getName, Func`1 getCustom, ICalibratorTrainerFactory calibrator, Int32 maxCalibrationExamples) in D:\a\1\s\src\Microsoft.ML.Data\EntryPoints\InputBase.cs:line 119
at Microsoft.ML.Trainers.Sdca.TrainBinary(IHostEnvironment env, Options input) in D:\a\1\s\src\Microsoft.ML.StandardTrainers\Standard\SdcaBinary.cs:line 2488
Standard Output Messages:
Test EntryPointChainedCrossValMacros: aborted: passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.