Answer on BBBC021 Holdout procedure

[jan-engelmann]

In the BBBC021 dataset, compounds are measured across different plates to account batch effect (technical variability of the cellular response to treatment). Each plate contains multiple wells, where different compounds are tested. Only one compound gets applied per well. Each one of the considered compounds is measured over multiple plates. For example, a drug like Cytochalasin B is measured in wells located on three different plates.

To ensure the model picks up the biological effect caused by compounds on cells, we evaluated the model by training it on two plates and testing its performance on a held-out one. This holding-out procedure is carried out on all the plates subsequentially, and the performance is average across them. So, for all drugs, we leave out one plate first, train the model on the remaining ones, and test on the leave-out plate. This is repeated for the other plates where the compounds are measured and the performances are averaged across drugs and hold-out experiments.