Feature request: calibration metrics

[UAB-BST-680]

I really like yardstick's API for computing discrimination. Calibration is also an important metric for classification models. Is there any interest in developing some yardstick functions to compute a calibration statistic?

Frank Harrell's Unreliability index seems like a good candidate. The Unreliability index is a chi-square statistic with 2 degrees of freedom testing the hypothesis that calibration slope and intercept are ideal (i.e., the slope is 1, the intercept is 0).

[alexpghayes]

On a related note, I would like to request calibration curves, as in, for example, https://scikit-learn.org/stable/auto_examples/calibration/plot_compare_calibration.html#sphx-glr-auto-examples-calibration-plot-compare-calibration-py

[ayueme]

Really need that toooo! Hope calibration curve can be easily plotted with yardstick! Thank you very much.

[Guiquan-27]

Yes!! This is also I really want. Thanks!

[aminadibi]

Calibration curves are the go-to metric in clinical prediction models, arguably more important that ROC curves.
See
https://link.springer.com/book/10.1007/978-3-030-16399-0