Benchmark for FastAMI - A Monte Carlo Approach to the Adjustment for Chance in Clustering Comparison Metrics
This repository contains the research code for our paper FastAMI - A Monte Carlo Approach to the Adjustment for Chance in Clustering Comparison Metrics which will be presented at AAAI-23 in February 2023. A standalone version of FastAMI for easier use in other projects is available at https://github.com/mad-lab-fau/fastami and can be installed from PyPi via pip install fastami. This repository is for archival purposes only.
This benchmark version compares our implementation with the AMI in scikit-learn, the pairwise AMI [3], and the SMI [1] and contains a preprocessed version of the Benchmark Suite for Clustering Algorithms - Version 1 [2].
To reproduce the results in our paper, you must first install Python 3.10.4 and the required dependencies:
pip install -r requirements.txt
For the direct SMI sampling, we use C code that must be compiled by executing.
fastami/rcont2/build.sh
For the clustered version of Benchmark Suite for Clustering Algorithms – Version 1 please unpack gagolewski.zip in
/data/gagolewski
For the synthetic EMI and SMI Benchmarks execute
python synthetic_benchmark.py
The benchmarks on real datasets can be executed as follows
python gagolewski_benchmark.py
and
python snap_benchmark.py
[1] S. Romano, J. Bailey, V. Nguyen, and K. Verspoor, “Standardized Mutual Information for Clustering Comparisons: One Step Further in Adjustment for Chance,” in Proceedings of the 31st International Conference on Machine Learning, Jun. 2014, pp. 1143–1151. Accessed: Dec. 08, 2021. [Online]. Available: https://proceedings.mlr.press/v32/romano14.html
[2] M. Gagolewski and others, “Benchmark Suite for Clustering Algorithms – Version 1.” 2020. doi: 10.5281/zenodo.3815066.
[3] D. Lazarenko and T. Bonald, “Pairwise Adjusted Mutual Information,” arXiv:2103.12641 [cs], Mar. 2021, Accessed: Sep. 16, 2021. [Online]. Available: http://arxiv.org/abs/2103.12641