Jump in Bioavailability_Ma Leaderboard AUROC Scores

[gozde-ozcan]

Hello,

The papers cited in the Bioavailability_Ma leaderboard report AUROC scores around 0.70 while the range in the leaderboard is currently between 0.80-0.95. As far as I can see, I haven't detected any major changes in the dataset itself. Is there an explanation for this discrepancy? Was there an update in the dataset and/or in the models?

[LeonardFH]

While I can't answer your question, I was wondering if it is possible to see on which date a certain model achieved its ranking on the Leaderboard? It would be great to know, since that would give an idea about the activity around ranking changes etc. I got the general sense the last time a new model moved into the Leaderboard was July 2024 from just anecdotal observations, which begs this question. Has the current models hit a plateau of achievement? Its like the 100 meter dash, eventually each sprinter clocks just under 10 seconds, but pushing past towards say 9 seconds, that would be phenomenal. I got the sense the best models strike out at around 0.7 with Murcko scaffolding in place, and without it clocking up anywhere from 0.9 to 0.96 on a good day. There is another angle that is currently not in measure it seems, that would be very cool to track, what is the minimum amount of training data you required for your model to hit say 0.9 ? Regards.