Leaderboard Question

[lan-lyu]

It says on the Leaderboard website that "To protect against overfitting on the blind test dataset you can only publish once every 7 days."
Does that mean we can only submit once since there are less than 7 days left?
I suppose there is no blind test dataset in the rearrangement task, so can we submit for many times?
Thanks!

[lan-lyu]

Also, is the json file the only thing we need to submit? (except those filled-in informations)
Many thanks ;)

[Lucaweihs]

Hi @Alyssa14411, I've went ahead and disabled the 7-day publishing bottleneck, feel free to submit several times in the next week. That said: please be mindful to not evaluate many models on the test set as this will eventually lead to overfitting on it.

Yes, the zipped json file is all you need to submit. We'll evaluate all the trajectories in the json file (which takes a few hours) to verify your results.

[lan-lyu]

Thanks a lot!!