XEvalAD

Requirements

1. Installing requirements

• Required libraries: numpy, pandas, seaborn, tensorflow, tqdm

• Alternatively, install all requirements using the provided requirements.txt file:

pip install -r requirements.txt

2. Preparing input data

XEvalAD recieves a csv file as input. This csv file should hold the measurments, e.g., ROC values corresponding to each algorithm-dataset pair. The first row should contain algorithms' names, and the first column should contain datasets' names.

We used the results from ADBench (Tables D4 and D10).

XEvalAD

• See XEvalAD_NoteBook_Main.ipynb

1. Import XEvalAD:

from XEvalAD import XEvalAD

2. Instantiate XEvalAD:

_PIJ_DATA_FILE_PATH = "./path/to/your/data/inpu_matrix.csv"
XEvalAD_instance = XEvalAD(PIJ_DATA_FILE_PATH=_PIJ_DATA_FILE_PATH)

3. Train XEvalAD from scratch or load learnt parameteres:

• To train XEvalAD:

XEvalAD_instance.fit_IRT_instance()

and save the latent traits for future use:

XEvalAD_instance.save_latent_traits_to_file()

Note: This function can receive an optional argument latent_traits_save_file_path. The default value is set to "latent_traits_save.npz".

• To load learnt traits from an existing npz file:

XEvalAD_instance.load_latent_traits_from_file()

Note: This function can receive an optional argument latent_traits_save_file_path. The default value is set to "latent_traits_save.npz".

4. Generate explainable analysis results:

Ability

For algorithms: XEvalAD_instance.plot_algorithms_abilities()

For datasets: XEvalAD_instance.plot_datasets_abilities()

Discrimination vs Difficulty and ICCs

For algorithms: XEvalAD_instance.plot_algorithms_scatter_icc()

For datasets: XEvalAD_instance.plot_datasets_scatter_icc()