PermutationFeatureImportance example with multiclassification

[drake7707]

The PFI extension also exists in the MultiClassClassification catalog but returns objects of MulticlassClassificationMetricsStatistics that do not have a RSquared property. As someone not intimately familiar with the ML terminology it is unclear to me which of the properties I should use (MacroAccuracy, MicroAccuracy, LogLoss, LogLossReduction, TopKAccuracy) to rank the features like in the example above.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 1b1fb0dd-648b-26e9-7e4e-48824ab94160
• Version Independent ID: 689b648-dadc-fd39-c432-ebf095a0299a
• Content: Explain model predictions using Permutation Feature Importance - ML.NET
• Content Source: docs/machine-learning/how-to-guides/explain-machine-learning-model-permutation-feature-importance-ml-net.md
• Product: dotnet-ml
• GitHub Login: @luisquintanilla
• Microsoft Alias: luquinta

[luisquintanilla]

Hi @drake7707 . That is correct. PFI is available for various machine learning tasks in ML.NET. The reason R-Squared is not available for multiclass classification is different machine learning tasks have different metrics. See this article on metrics for more details.

https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/metrics

[drake7707]

@luisquintanilla Thanks, I completely missed that page.

Can I interpret the MacroAccuracy,MicroAccuracy and LogLoss in the same way as RSquared, are they also average decreases per feature? If so, am I correct that I should just replace the RSquared.Mean from the example with LogLoss.Mean to rank the feature importance correctly?

[luisquintanilla]

@drake7707 That is correct. See sample from API documentation

[drake7707]

I wasn't aware there were examples documented in the API documentation. My search-fu failed me, thanks a lot!