cross_val_score threshold adjustment

[tjdurant]

Apologies if this has been brought up before but a search through current and closed issues didn't return anything to indicate it had.

I have a use case wherein I have picked an optimal decision threshold based on the intersection of sensitivity and PPV when plotted as a function of the decision threshold. The point of intersection corresponds with a decision threshold of 0.3. I would like to run cross_val_score with a LogisticRegression model that uses a threshold of 0.3 to calculate a distribution of sensitivity and FPR (separate issue, here). However, it doesn't seem as though the threshold can be varied in the context of cross_val_score or at the estimator level. Might be nice to have that functionality if possible.

Thanks to all the contributors who work on this project : )

-Tommy

[amueller]

That's not currently possible out-of-the-box.
You can write a meta-estimator to achieve that yourself, though (i.e. something that wraps the LogisticRegression and changes the threshold).
There is a more general solution in the works here: #10117

[cmarmo]

Cross referencing here #16525 , as it is meant to supersede #10117. @glemaitre do you think your PR will also close this issue? Thanks.

[glemaitre]

This is a use-case mentioned in #16525 (still not implemented thought) by @lorentzenchr. It might make sense to add the option and it would close this issue as well.