EvalAI is a Django-based application, hence it leverages the concept of Django apps to properly namespace the functionalities. All the apps can be found in the apps
directory situated in the root folder.
Some important apps along with their main uses are:
- Challenges
This app handles all the workflow related to creating, modifying, and deleting challenges.
- Hosts
This app is responsible for providing functionalities to the challenge hosts/organizers.
- Participants
This app serves users who want to take part in any challenge. It contains code for creating a Participant Team, through which they can participate in any challenge.
- Jobs
One of the most important apps, responsible for processing and evaluating submissions made by participants. It contains code for creating a submission, changing the visibility of the submission and populating the leaderboard for any challenge.
- Web
This app serves some basic functionalities like providing support for contact us or adding a new contributor to the team, etc.
- Accounts
As the name indicates, this app deals with storing and managing data related to user accounts.
- Base
A placeholder app which contains the code that is used across various other apps.
Settings are used across the backend codebase by Django to provide config values on a per-environment basis. Currently, the following settings are available:
- dev
Used in development environment
- testing
Used whenever test cases are run
- staging
Used on staging server
- production
Used on production server
The base URLs for the project are present in evalai/urls.py
. This file includes URLs of various applications, which are also namespaced by the app name. So URLs for the challenges
app will have its app namespace in the URL as challenges
. This actually helps us separate our API based on the app.
The whole codebase for the frontend resides in a folder named frontend
in the root directory
Scripts contain various helper scripts, utilities, python workers. It contains the following folders:
- migration
Contains some of the scripts which are used for one-time migration or formatting of data.
- tools
A folder for storing helper scripts, e.g. a script to fetch pull request
- workers
One of the main directories, which contains the code for submission worker. Submission worker is a normal python worker which is responsible for processing and evaluating submission of a user. The command to start a worker is:
python scripts/workers/submission_worker.py
All of the codebase related to testing resides in tests
folder in the root directory. In this directory, tests are namespaced according to the app, e.g. tests for challenges
app lives in a folder named challenges
.
To perform certain actions like seeding the database, we use Django management commands. Since the management commands are common throughout the project, they are present in base
application directory. At the moment, the only management command is seed
, which is used to populate the database with some random values. The command can be invoked by calling
python manage.py seed