Running online (on mybinder.org)

Click on Binder icon below.

After some time (it may take up to 2-3 minutes, be patient) binder will load the environment and you can use familiar Jupyter interface. For example, clicking on ICLR_Reviews.ipynb, you will open the notebook with the analysis of ICLR reviews. Note that in the tools like nbviewer or colab qgrid library does not display the tables, so that's why by default the table in the notebook is not visible, and you have to generate the table by running the cells yourself.

Running locally

First, install requirements:

• openreview
• numpy
• pandas
• qgrid (better via conda, consult here for pip installation)
• jellyfish (via pip)
• matplotlib

Then, download the latest version of the notebook and run it as usual.

Highlights

Currently analysis includes:

• Interactive table with all reviews for all papers, which you can sort or filter by the topic, rating, or any other column.

• Frequency by topic.

• Heatmap of confidence over the reviews.

• Number of reviews.

• Raw statistics on the reviews such as average review and counts of each score.

Feedback

Any suggestions, questions, and contributions are welcome.