TinyCP is an experimental Python library for conformal predictions, providing tools to generate valid prediction sets with a specified significance level (alpha). This project aims to facilitate the implementation of personal and future projects on the topic.
For more information on a previous project related to Out-of-Bag (OOB) solutions, visit this link.
calibrate: instead ofBalanced Accuracy Score, it can be calibrated eitherMatthews Correlation CoefficientorBookmaker Informedness Score, for better reliability.evaluate: scoresbmandmccfor more reliability.
Currently, TinyCP supports Out-of-Bag (OOB) solutions for RandomForestClassifier in binary classification problems, as well as RandomForestRegressor and RandomForestQuantileRegressor for regression tasks. For additional options and advanced features, you may want to explore Crepes.
Install TinyCP using pip:
pip install tinycpNote: If you want to enable plotting capabilities, you need to install the extras using Poetry:
poetry install --E plotImport the conformal classifiers from the tinycp.classifier module:
from tinycp.classifier import BinaryClassConditionalConformalClassifier
from tinycp.classifier import BinaryMarginalConformalClassifierImport the conformal regressors from the tinycp.regressor module:
from tinycp.regressor import ConformalizedRegressor
from tinycp.regressor import ConformalizedQuantileRegressorExample usage of BinaryClassConditionalConformalClassifier:
from sklearn.ensemble import RandomForestClassifier
from tinycp.classifier import BinaryClassConditionalConformalClassifier
# Create and fit a RandomForestClassifier
learner = RandomForestClassifier(n_estimators=100, oob_score=True)
X_train, y_train = ... # your training data
learner.fit(X_train, y_train)
# Create and fit the conformal classifier
conformal_classifier = BinaryClassConditionalConformalClassifier(learner)
conformal_classifier.fit(y=y_train, oob=True)
# Make predictions
X_test = ... # your test data
predictions = conformal_classifier.predict(X_test)Evaluate the performance of the conformal classifier using the evaluate method:
results = conformal_classifier.evaluate(X_test, y_test)
print(results)BinaryMarginalConformalClassifier A marginal coverage conformal classifier methodology utilizing a classifier as the underlying learner. This classifier supports the option to use Out-of-Bag (OOB) samples for calibration.
BinaryClassConditionalConformalClassifier A class conditional conformal classifier methodology utilizing a classifier as the underlying learner. This classifier supports the option to use Out-of-Bag (OOB) samples for calibration.
ConformalizedRegressor A conformal regressor methodology utilizing a regressor as the underlying learner. This regressor supports the option to use Out-of-Bag (OOB) samples for calibration, providing valid prediction intervals for regression tasks.
ConformalizedQuantileRegressor A conformal quantile regressor methodology utilizing a quantile regressor as the underlying learner. This regressor supports the option to use Out-of-Bag (OOB) samples for calibration, offering more robust prediction intervals by leveraging quantile estimates.
This project is licensed under the MIT License.