Why does TestRocAuc take so long?

[ozeshun]

Hello, I am using evidently on developing MLOps base, and facing the issue of slow execution time of TestRocAuc (The data inputting for TestRocAuc has about 1 million rows).
Could you please give me some solutions?

What I verified

• I set TestRocAuc thresholds, but execution time did not improve reference.
• There is no parameter in QualityMetricsOptions that improves execution time.
• It looks like scikit-learn.metrics.roc_curve is used when calculating AUC (Calculations using scikit-learn API should be fast) reference.

[feldlime]

I faced the same problem and did some investigations.

Calculation of roc-auc invokes 6 calls of the calculation method of different classes. One of these classes is ClassificationQualityMetric which calls evidently.calculations.classification_performance.calculate_metrics function. This function calculates all basic classification metrics.

First point here that you don't actually need all classification metrics to get roc-auc value.

Second, and the most important point is that the code is very inefficient in these lines where for every predicted value some calculations are made using pandas.

[elenasamuylova]

Hi @feldlime, thank you for taking a look! Would you want to contribute an optimization?

[feldlime]

Hi @elenasamuylova, I can try :)