AutoML for Outlier Detection with Optimal Transport Distances (IJCAI2023)

[prabhant]

TLDR: We used GWLR implementation from OTT-JAX to estimate dataset similarity between unsupervised tabular datasets to recommend outlier detection algorithms.

Abstract: Automated machine learning (AutoML) has been widely researched and adopted for supervised problems, but progress in unsupervised settings has been limited. We propose `"LOTUS", a novel framework to automate outlier detection based on meta-learning. Our premise is that the selection of the optimal outlier detection technique depends on the inherent properties of the data distribution. We leverage optimal transport to find the dataset with the most similar underlying distribution, then apply the outlier detection techniques that proved to work best for that data distribution. We evaluate the robustness of our framework and find that it outperforms all state-of-the-art automated outlier detection tools. This approach can also be easily generalized to automate other unsupervised settings.
Paper link: https://www.ijcai.org/proceedings/2023/843

[marcocuturi]

Hi Prabant, looks cool, maybe a link to the paper or pdf?

[prabhant]

This is the workshop preprint from NeurIPS, the IJCAI paper will be available soon.
https://arxiv.org/abs/2211.00372

[prabhant]

We recently used a similar technique for Automated clustering on tabular data and got SOTA results.