Results of the paper "Explanation-by-Example based on Item Response Theory"
This repository is organized as follows:
- decodIRT_OtML_mod.py: Script of the decodIRT tool that was modified to suit the interests of this research.
- dataset_properties.csv: File containing the 15 properties of the 4 datasets that were used as a case study.
- decodIRT_output: Folder containing all the files generated by the decodIRT tool.
- results: Folder containing other results that were generated during the experiments.
In the decodIRT_output directory, there is a folder for each dataset that contains the following files:
- dataset_name.csv: The file without suffix indicates that its content is the answer of each of the ML models of the real classifiers and the artificial classifiers.
- dataset_name_acuracia.csv: The file with the suffix “_acuracia” means that its content is composed of a table containing the average accuracy of each real classifier, during cross-validation.
- dataset_name_final.csv: The file with the suffix “_final” means that its content consists of a table containing the accuracy of the real classifiers on the separate instances for testing.
- dataset_name_irt.csv: Just like the file without suffix in the name (dataset_name.csv), this file has an array of responses. However, it does contain a response vector for all real, artificial and RF classifiers.
- dataset_name_mlp.csv: Contains the final accuracy that the first set RF's classifiers obtained after the classification.
- irt_item_param.csv: Table containing all item parameters (Difficulty, Discrimination and Guessing) generated for the test set instances.
- dataset_name_train.csv: Contains the set of dataset instances used to train the models.
- dataset_name_test.csv: Contains the set of dataset instances used to test the models.
- models: Contains all ML models that were generated, saved in .pkl format.
In the results directory, there is a folder for each dataset that contains the following files:
- corrInst_DisNeg.csv: Correlation of features with negative discrimination, considering instances with negative discrimination.
- corrInst_Erro.csv: Correlation of features with item parameters, considering the instances that the analyzed model failed.
- corrInst_ItemParam.csv: Correlation of features with item parameters, considering all test instances.
- heatInst_DisNeg.png: Heatmap of the correlation of features with negative discrimination.
- heatInst_error.png: Heatmap of the correlation of features with item parameters, considering the instances that the analyzed model failed.
- heatInst_ItemParam.png: Heatmap of the correlation of features with item parameters, considering all test instances.
- ItemParam_3D.png: 3D scatter graph of the relationship between dataset instances arranged for each item parameter and divided by class.