- No network configuration required.
- Does not depend on external platforms like e.g. Kaggle.
- Can be easily installed on a personal computer.
- Provides a customizable template in english and french.
Further documentation is available in the Reference manual.
Please report bugs, troubles or discussions on the Issues tracker. Any contribution to improve the package is welcome.
Install the R package from CRAN repositories
or install the latest development version from GitHub
# install.packages("devtools") devtools::install_github("adrtod/rchallenge")
Install a new challenge in
setwd("~/Dropbox/mychallenge") library(rchallenge) ?new_challenge new_challenge()
or for a french version:
new_challenge(template = "fr")
You will obtain a ready-to-use challenge in the folder
challenge.rmd: template R Markdown script for the webpage.
data: directory of the data containing
submissions: directory of the submissions. It will contain one subdirectory per team where they can submit their submissions. The subdirectories are shared with Dropbox.
history: directory where the submissions history is stored.
The default challenge provided is a binary classification problem on the German Credit Card dataset.
You can easily customize the challenge in two ways:
- During the creation of the challenge: by using the options of the
- After the creation of the challenge: by manually replacing the data files in the
datasubdirectory and the baseline predictions in
submissions/baselineand by customizing the template
To complete the installation:
Create and share subdirectories in
submissionsfor each team:
?new_team new_team("team_foo", "team_bar")
Render the HTML page:
output_dirargument to change the output directory. Make sure the output HTML file is rendered, e.g. using GitHub Pages.
Give the URL to your HTML file to the participants.
Refresh the webpage by repeating step 2 on a regular basis. See below for automating this step.
From now on, a fully autonomous challenge system is set up requiring no further administration. With each update, the program automatically performs the following tasks using the functions available in our package:
store_new_submissionsreads submitted files and save new files in the history.
print_readerrdisplays any read errors.
compute_metricscalculates the scores for each submission in the history.
get_bestgets the highest score per team.
print_leaderboarddisplays the leaderboard.
plot_historyplots a chart of score evolution per team.
plot_activityplots a chart of activity per team.
Automating the updates
You can setup the following line to your crontab using
crontab -e (mind the quotes):
0 * * * * Rscript -e 'rchallenge::publish("~/Dropbox/mychallenge/challenge.rmd")'
This will render a HTML webpage every hour.
output_dir argument to change the output directory.
If your challenge is hosted on a Github repository you can automate the push:
0 * * * * cd ~/Dropbox/mychallenge && Rscript -e 'rchallenge::publish()' && git commit -m "update html" index.html && git push
You might have to add the path to Rscript and pandoc at the beginning of your crontab:
Depending on your system or pandoc version you might also have to explicitly add the encoding option to the command:
0 * * * * Rscript -e 'rchallenge::publish("~/Dropbox/mychallenge/challenge.rmd", encoding = "utf8")'
You can use the Task Scheduler to create a new task with a Start a program action with the settings (mind the quotes):
- The rendering of HTML content provided by Dropbox will be discontinued from the 3rd October 2016 for Basic users and the 1st September 2017 for Pro and Business users. See https://www.dropbox.com/help/16. Alternatively, GitHub Pages provide an easy HTML publishing solution via a simple GitHub repository.
- version 1.16 of pandoc fails to fetch font awesome css, see https://github.com/jgm/pandoc/issues/2737.
Credit approval (in french) by Adrien Todeschini (Bordeaux).
Spam filter (in french) by Marie Chavent (Bordeaux).
Please contact me to add yours.
Copyright (C) 2014-2015 Adrien Todeschini.
Contributions from Robin Genuer.
Design inspired by Datascience.net, a french platform for data science challenges.
The rchallenge package is licensed under the GPLv2 (https://www.gnu.org/licenses/gpl-2.0.html).
To do list
- do not take baseline into account in ranking
- examples, tests, vignettes
- interactive plots with
- check arguments
- interactive webpage using Shiny