Feature Request: Some Metric(s) for Prediction Quality/Accuracy/Goodness/RELIABILITY of every SINGLE Prediction

[petersesztak]

It's should be very valuable to get Some Metric(s) for Prediction Quality/Accuracy/Goodness/Any of every SINGLE Prediction.

Background: when a prediction engine works in a real production enviroment it's should be great value to get infomation about Quality/Accuracy/Goodness/AnyRelevantMetric of every SINGLE Prediction.

Something like that:
class HousingPrediction
{
[ColumnName("Score")]
public float PredictedPrice { get; set; }

public float **ProbabilityOfPredictionAccuracy** { get; set; }

}

Use case: when consuming a given prediction it almost so important to know not the PREDICTION VALUE only, but the RELIABILITY of that PREDICTION.

Best Regards,
Péter

[rosebyte]

ML.NET cannot provide a model-agnostic reliability value for an individual prediction. Classification trainers may expose calibrated probabilities, while regression uncertainty requires model-specific methods and assumptions.