UQ Methods to add

[nilsleh]

This issue can be a collection and discussion of methods we could add to the library at some point, in no particular order :) Feel free to comment with suggestions and if you feel comfortable, you are more than welcome to add them via a PR!

An overview of some methods from the survey paper by Gawlikowski et al. 2023 is here.

Methods remaining to be grouped

• Stein Variational Gradient Descent
• Density Uncertainty Layers
• Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent
• CARD: Classification and Regression Diffusion Models (Implemented in Implement CARD Regression #47) but missing support for classification
• Epistemic Neural Networks
• Prediction-powered inference
• Collapsed Inference for Bayesian Deep Learning
• Posterior Refinement Improves Sample Efficiency in BNNs
• Quantification of Uncertainty with Adverserial Models
• Single Model Uncertainty
• Repulsive Deep Ensembles
• Neural Diffusion Processes
• DropConnect
• Probabilistic Circuits
• Single-shot Bayesian Approximation for NN Code
• Implicit Variational Inference for High-Dimensional Posteriors
• Convolutional Deep Kernel Machine
• Spline Prediction Intervals via Conformal Estimation (SPICE)
• Evidential Deep Learning: Predictive Uncertainty for Earth System Modelling
• Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models
• SNGP Add SNGP UQ-Method #98
• Direct Epistemic Uncertainty Prediction (DEUP)
• MixupMP
• EasyUQ
• Lightweight Probabilistic Networks

Methods by Task

Regression:

• Deep Evidential Regression for Bayesian Quantile Regression
• Autoregressive Conditional Neural Process
• Convolutional Conditional Neural Process
• Wasserstein Generative Regression
• Escaping the Sampling Trap: Fast and Accurate Epistemic Uncertainty Estimation with Pairwise-Distance Estimators
• Variational Bayesian Last Layer

Classification:

• DropMax
• Deterministic Uncertainty Quantification (DUQ)
• Deep Deterministic Uncertainty
• Deep Evidential Classification
• Unified Uncertainty Calibration
• Dirichlet Based Uncertainty Models

PixelWise Regression:

• BayesCap
• Marigold possibly interesting EO application, intended for Monocular Depth Estimation
• Image2Image Regression with Code Add Img2Img Conformal Prediction #118
• Deep Energy Based Models
• Normalizing Flows

Segmentation:

• Probabilistic Unet
• Segmentation Diffusion
• Deep Deterministic Uncertainty

Generative Models:

• LatentDiffusion Model and with UQ application

Post-hoc calibration procedures:

• RASP Conformal (Add Regularized Adaptive Prediction Sets (RAPS) UQ Method #57)
• Temperature Scaling (Add UQ-method Temperature Scaling #56)
• Conformal Risk Control
• Conditional Conformal Prediction
• SASP Conformal
• MC-Dropout Conformal
• Monte Carlo Conformal

[wrkhard]

I would be very interested in seeing SNGP implemented for ViTs. Seems to be very few UQ methods out there for ViT.

Paper: https://papers.nips.cc/paper/2020/file/543e83748234f7cbab21aa0ade66565f-Paper.pdf
TF implementation for ResNet: https://www.tensorflow.org/tutorials/understanding/sngp#full-sngp-model

[qqaazz800624]

Thanks for your hard work!
I just came across a 2024 ICLR paper a few days ago which I found it quite interesting and encouraging. The paper proposed BayesDiff, a pixel-wise uncertainty estimator for generations from diffusion models based on Bayesian inference.
Since Diffusion models has gained so much attention and proved to have impressive image generation capability, I am glad to know that someone has found a way to combine uncertainty quantification and Diffusion models.

KOU, Siqi, et al. BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference. arXiv preprint arXiv:2310.11142, 2023.

The Github:
https://github.com/karrykkk/BayesDiff