Project_CodaLab_FAQ
CodaLab competitions is a web-based interactive computing environment that allows researchers to share and interact with code, data, and experiments in the context of online competitions. CodaLab is free and open-source (you can get the source code on Github). We run a free public instance: https://competitions.codalab.org/.
CodaLab competitions is aimed at researchers, scientists, and other professionals who want to further technology in their fields by hosting and/or participating in algorithm competitions.
A competition consists of data for training and evaluating algorithms, and a program to evaluate participant results. CodaLab competitions provide an opportunity for researchers, developers and algorithmists to create solutions for problems across a wide range of domains, and advance the state of the art for their respective areas of interest.
No. Anybody can use the public instance of Codalab competitions for free, as long as he/she agrees with the Terms and conditions. However, if you run a large competition with code submission, we advise you to first contact us. To find out how to host your own competition, go to the competition help page.
The CodaLab platform is built using Python, Django and Foundation.
Yes, you can host your own CodaLab instance on a private or hosted server (e.g. Azure or AWS). Some customization is possible as well. For more information, see Hosting your own instance. However, most competition organizers do NOT need to run their own instance. If you run a computationally demanding competition, you can hook up your own compute workers in the backend very easily. See Using your own compute workers.
To see a current list of competitions hosted on the public instance of Codalab competitions, go to the competitions page. Currently there is no way of tracking competitions hosted on private instances, but this will come with version 2.
When you submit code to the Codalab platform, your code is executed inside a docker container. This environment can be exactly reproduced on your local machine by downloading the corresponding docker image. The Codalab docker environment contains a large number of pre-loaded programs, such as Python. You can download or folk the default image docker-codalab-legacy-worker or get it from docker hub:codalab/codalab-legacy. If you are new to docker, follow these instructions to install docker. You can quickly get up and running by following the cheat sheet. If you are an organizer, you can specify your new docker as a competition property in the competition YAML file or the editor. Contact us if you need participants to specify their own docker.
Codalab does not forbid the use of robots (bots) to access the website, provided that it is not done with malicious intentions to disturb the normal use and jam the system. A user who abuses their rights by knowingly, maliciously, or recklessly jamming the system, causing the system to crash, causing loss of data, or gaining access to unauthorized data, will be banned from accessing all Codalab services.