Numerical columns missing

[iluveu28]

System information

• OS version/distro: Linux
• .NET Version (eg., dotnet --info): 3.1

Issue

This is a follow to the previous issue (#5459) that @antoniovs1029 previously helped with.

I have a project where the training data has 27 columns. Somehow the InferColumns excluded all the numerical columns.

    var columnInference = _mlContext.Auto().InferColumns(trainingFilePath, labelColumnName);
    var textLoader = _mlContext.Data.CreateTextLoader(columnInference.TextLoaderOptions);

    var trainingDataView = textLoader.Load(trainingFilePath);

When I inspect the columnInference variable, I see only the string columns. Why?

Also, when I inspect the trainingDataView variable, I see an additional column created called Features which is a vector type.

Although the training was successful and saved. The prediction is not working because it's complaining the Features column is missing. How do I get around this?

[jwood803]

Hey, @iluveu28, hope you don't mind me chiming in here.

The InferColumns method groups the columns into a Features column, but that can be worked around by setting the groupColumns parameter as false. This will keep the column names so you can use them later for predictions.

var columnInformation = mlContext.Auto().InferColumns(trainingFilePath, labelColumnName,
  groupColumns: false);

Hope that helps!

[iluveu28]

@jwood803 thanks for chiming in.

First of all, why does it group the columns in the first place ? It doesn't group the numerical columns for my other project which has only 5 columns in total. How does it determine when to group them?

So the missing Features column error is gone now after setting the flag to false, thank you. However, the results are empty. The predictions variable is returning a list of 0 values. What am I doing wrong below? Again, the same code works fine for my other project with 5 columns.

	var options = new TextLoader.Options
	{
		Separators = new[] { ',' },
		HasHeader = true,
		Columns = columns,
		AllowQuoting = true,
		AllowSparse = true
	};

	var loader = _mlContext.Data.CreateTextLoader(columns,
			hasHeader: true,
			separatorChar: ',');

            File.WriteAllText(testDataPath, content);

            var testData = loader.Load(testDataPath);

            ITransformer mlModel = _mlContext.Model.Load(modelPath, out var modelInputSchema);

            var output = mlModel.Transform(testData);

            var predictions = output.GetColumn<UInt32>(labelColumnName);

            var predictedLabel = output.GetColumn<ReadOnlyMemory<char>>("PredictedLabel");

            int count = predictedLabel.Count();

            string jsonResults = "";

            foreach (ReadOnlyMemory<char> result in predictedLabel)
	{
              jsonResults += "{ output: " + result.ToString() + "}";

	}

[iluveu28]

@michaelgsharp @jwood803 would really appreciate a swift response.