{{ message }}

MATLAB code to calculate a new version of the Signed Rank Test for comparing two classifiers on multiple datasets, as explained in a paper by Benavoli et al. (2017). The code is adapted from the python library baycomp.

# LucyKuncheva/Bayesian-Analysis-for-Comparing-Classifiers

Switch branches/tags
Nothing to show

## Files

Failed to load latest commit information.
Type
Name
Commit time

# Bayesian-Analysis-for-Comparing-Classifiers

This is a small MATLAB implementation of some of the functionalities of Janez Demsar’s Python library `baycomp` available here: https://github.com/janezd/baycomp. The code can be used for comparing two classifiers using the Signed Rank Test proposed in the paper: A. Benavoli, G. Corani, J. Demsar, M. Zaffalon, "Time for a Change: a Tutorial for Comparing Multiple Classifiers Through Bayesian Analysis", Journal of Machine Learning Research 18 (2017) 1-36.

The repository contains the following files:

• `BayesianAnalyisExample.m` This is an example that runs the Bayesian Analysis exaplained in the paper and reproduces

• `NBC_AODE_differences.xlsx` This is an excel file with the differences between classifiers NBC (Naive Bayes) and AODE (averaged one-dependence estimator). The values are taken from Table 7 in the paper.

• `signed_rank_test_diff.m` This is a function that takes a vector of differences of classifier accuracies across multiple data sets (A-B) and calculates the results of the signed rank test. The function is largely translated from the respective component of the `baycomp` library. It outputs an array with probability density values (theta_left, theta_rope, theta_right) needed for the triangle plot and the set of three probabilities p_left (classifier A is better), p_rope (the two classifiers are equivalent) and p_right (classifier B is better).

• `barycentric_plot.m` This code is used to plot the triangle diagram with the cloud of simulated points in barycentric coordinates.

• `density_plot.m` This is a function that produces a scatterplot in the current axes, where each point is coloured depending on the density of its neighbourhood.

• `two_classifiers_one_dataset` This function computes probabilities using a Bayesian correlated t-test. It was written directly in MATLAB using the text of the paper [Benavoli et al., 2017]. The input is a vector of differences of classifier accuracies across one or repeated cross-validation experiment (A-B). The function outputs an array with probabilities p_left (classifier A is better), p_rope (the two classifiers are equivalent) and p_right (classifier B is better). It also outputs the x-y values for the density plot.

``````@misc{KunchevaMATLABBayesian2020,
author = {Ludmila I Kuncheva},
title = {Bayesian-Analysis-for-Comparing-Classifiers},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/LucyKuncheva/Bayesian-Analysis-for-Comparing-Classifiers}}
}
``````

MATLAB code to calculate a new version of the Signed Rank Test for comparing two classifiers on multiple datasets, as explained in a paper by Benavoli et al. (2017). The code is adapted from the python library baycomp.

## Releases

No releases published

## Packages 0

No packages published