Does ML.NET support multi-label classification

[Terrance-Whitehurst]

I am currently using ML.net for multi-class text classification. I was wondering is their a way to do multi-label text classification in ML.NET

[luisquintanilla]

Hi @Terrance-Whitehurst,

It does, though not out of the box. One way to do this is by mapping the labels and their scores.

This issue has the code on how to perform that mapping:

dotnet/machinelearning-modelbuilder#2399 (comment)

        /// <summary>
        /// Map the unlabeled result score array to the predicted label names.
        /// </summary>
        /// <param name="result">Prediction to get the labeled scores from.</param>
        /// <returns>Ordered list of label and score.</returns>
        /// <exception cref="Exception"></exception>
        public static IOrderedEnumerable<KeyValuePair<string, float>> GetSortedScoresWithLabels(ModelOutput result)
        {
            var unlabeledScores = result.Score;
            var labelNames = GetLabels(result); 

            Dictionary<string, float> labledScores = new Dictionary<string, float>();
            for (int i = 0; i < labelNames.Count(); i ++)
            {
                // Map the names to the predicted result score array
                var labelName = labelNames.ElementAt(i);
                labledScores.Add(labelName.ToString(), unlabeledScores[i]);
            }

            return labledScores.OrderByDescending(c => c.Value);
        }

        /// <summary>
        /// Get the ordered label names.
        /// </summary>
        /// <param name="result">Predicted result to get the labels from.</param>
        /// <returns>List of labels.</returns>
        /// <exception cref="Exception"></exception>
        private static IEnumerable<ReadOnlyMemory<char>> GetLabels(ModelOutput result)
        {
            var schema = PredictEngine.Value.OutputSchema;

            var scoreColumn = schema.GetColumnOrNull(nameof(result.Score));
            if (scoreColumn == null)
            {
                throw new Exception("Score column not found. Make sure the name searched for matches the name in the schema.");
            }

            // SlotNames are a column annotation. In this case, it contains an ordered array of the possible labels.
            // The ML.NET result order is the same as the order in slot names. This allows us to map the name to the results.
            var slotNames = new VBuffer<ReadOnlyMemory<char>>();
            scoreColumn.Value.GetSlotNames(ref slotNames);
            return slotNames.DenseValues();
        }

Closing this issue for now. Feel free to file a new issue if you have additional questions / issues.

[Terrance-Whitehurst]

@luisquintanilla Thank you so much for your answer! Is their currently a way to do this in the model builder? And is the consumption file in that issue for a console application?

[luisquintanilla]

@Terrance-Whitehurst Yes? I'll let @beccamc confirm since I forget whether we already shipped this in Model Builder or it's coming in the next release in a few days.

[beccamc]

@Terrance-Whitehurst The latest Model Builder (17.14.3.2310103) should include a "GetSortedScoresWithLabels" method in the consume code for multi-class scenarios.

[Terrance-Whitehurst]

@beccamc Thanks for this will check this out! What would be the best way to set up a dataset for multi-label classification for ML.NET? Right now I have two columns.

1. Text input to classify
2. A list of labels for each text entry

Is there a better way to setup the dataset?

[Terrance-Whitehurst]

@beccamc Thanks for this will check this out! What would be the best way to set up a dataset for multi-label classification for ML.NET? Right now I have two columns.

1. Text input to classify

2. A list of labels for each text entry

Is there a better way to setup the dataset?

@beccamc Just wanted to follow up on this.

[luisquintanilla]

@Terrance-Whitehurst you might want to structure this as:

TextInput	Label
Input1	Label1
Input1	Label2
Input2	Label3

I don't believe our classification scenarios support a label vector / array.