Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shuffle input cursor reader failed with an exception in ml.net 1.4.0 #4819

Closed
Spocher opened this issue Feb 8, 2020 · 3 comments
Closed
Labels
bug Something isn't working classification Bugs related classification tasks need info This issue needs more info before triage P1 Priority of the issue for triage purpose: Needs to be fixed soon.

Comments

@Spocher
Copy link

Spocher commented Feb 8, 2020

  • Windows 10 x64
  • NET Framework 4.7.2, ML.Net 1.4.0

I create project for multiclass classification, some code:
`var trainingDataView = mlContext.Data.LoadFromEnumerable(allTrainData);

var trainingPipeline = mlContext.Transforms.Conversion.MapValueToKey(inputColumnName: "IdCategory", outputColumnName: "Label")
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "HeaderProduct", outputColumnName: "HeaderProductFeaturized"))
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "DescriptionProduct", outputColumnName: "DescriptionProductFeaturized"))
.Append(mlContext.Transforms.Concatenate("Features", "HeaderProductFeaturized", "DescriptionProductFeaturized"))
.Append(mlContext.MulticlassClassification.Trainers.SdcaMaximumEntropy())
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));

var trainedModel = trainingPipeline.Fit(trainingDataView);`

I try train model and have Exception:
Shuffle input cursor reader failed with an exception, ( in Microsoft.ML.Transforms.RowShufflingTransformer.Cursor.MoveNextCore()
in Microsoft.ML.Data.RootCursorBase.MoveNext()
in Microsoft.ML.Trainers.TrainingCursorBase.MoveNext()
in Microsoft.ML.Trainers.SdcaTrainerBase3.TrainCore(IChannel ch, RoleMappedData data, LinearModelParameters predictor, Int32 weightSetCount) in Microsoft.ML.Trainers.StochasticTrainerBase2.TrainModelCore(TrainContext context)
in Microsoft.ML.Trainers.TrainerEstimatorBase2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) in Microsoft.ML.Data.EstimatorChain1.Fit(IDataView input)

I downgraded the version ml.net to 1.3.1 an all worked fine.
a little later, I upgraded the version to 1.5.0-preview and all worked fine.

@frank-dong-ms-zz frank-dong-ms-zz added bug Something isn't working P1 Priority of the issue for triage purpose: Needs to be fixed soon. labels Feb 10, 2020
@frank-dong-ms-zz
Copy link
Contributor

frank-dong-ms-zz commented Feb 10, 2020

Can you please provide a repro project and input data so we can look further?

I have also checked the release notes, seems nothing relate to this issue I can find.

@frank-dong-ms-zz frank-dong-ms-zz added the need info This issue needs more info before triage label Feb 11, 2020
@harishsk harishsk added the classification Bugs related classification tasks label Apr 29, 2020
@frank-dong-ms-zz
Copy link
Contributor

We didn't hear back for couple months now, so I will close the issue for now, @Spocher feel free to reopen this issue if you can provide more details on this issue, thanks.

@ScubaAddict1
Copy link

I had the same issue, fixed it by adding
_trainingDataView = _mlContext.Data.Cache(_trainingDataView);
Not sure if its ideal as am using a asp.net core (.net 5) website with ML 1.6 and loading the data from SQL.

@dotnet dotnet locked as resolved and limited conversation to collaborators Mar 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working classification Bugs related classification tasks need info This issue needs more info before triage P1 Priority of the issue for triage purpose: Needs to be fixed soon.
Projects
None yet
Development

No branches or pull requests

4 participants