EdX Notebook Grader

Main repository for the External Grade Server used by the Quantum Computer Systems Design course.

Submodule repos:

• jupyter_grade_server
• dockernel
• lab notebook sources (private)

Install and Run

Install:

git clone https://github.com/cduck/edx_notebook_grader
cd edx_notebook_grader
git submodule init
git submodule update
./install.sh

Create and edit notebooks (one per assignment):

./edit-notebooks.sh
./regenerate-notebooks.sh

Run the server (make sure this is always running, otherwise submissions won't get graded):

./run-server.sh

EdX Studio Configuration

To use this grader, you must also configure the problem submission page in your EdX course.

How to add the problem submission component:

1. Edit the course with studio.edx.org.
2. Go to a subsection and select "Add New Component" > "Problem" > "Advanced" > "Blank Advanced Problem"
3. Click "EDIT" and paste in the below XML:

   <problem>
     <coderesponse queuename="uchicago-qcs-xqueue">
       <label>Submit your completed IPython notebook (.ipynb file) for Lab X here.  Feedback and any error messages will be shown below.  It may take a few minutes to grade.</label>
       <filesubmission id="notebook" allowed_files="labX.ipynb" required_files="labX.ipynb"/>
       <codeparam>
         <grader_payload>
           {"name": "labX"}
         </grader_payload>
       </codeparam>
     </coderesponse>
   </problem>

4. Remember to replace labX with the actual assignment name (4 locations in the XML) and uchicago-qcs-xqueue with the course's XQueue name.
5. Switch from the "EDITOR" to "SETTINGS" and set the "Problem Weight" to 100.
6. "Publish" the subsection and "View Live Version" to test the submission process. (Clicking "Submit" when viewing the problem in studio.edx.org doesn't work.)

See the EdX docs on external graders for more information.

File structure for grader-root

• edx_notebook_grader/
  • grader-root/
    • conf.d/
      • 600.json
    • relocate/
      • nbgrader_config.py (copy of edx_notebook_grader/nbgrader_config.py)
      • source/ (see nbgrader docs)
        • header.ipynb
        • assignment1-name
          • assignment1-name.ipynb
        • assignemtn2-name
          • assignment2-name.ipynb
        • ...
      • release/ (generated and used by the server)
      • gradebook.db (generated and used by the server)
  • ...

nbgrader Notebook Requirements

nbgrader allows you (the instructor) to mark special cells in a notebook as "solutions" or "tests". When nbgrader generates the release version of the notebook, it replaces instructor solutions with # YOUR CODE HERE and removes your hidden test code.

Example source/lab1/lab1.ipynb:

(cell metadata: ID=1-2-3, Autograded answer)

# Question 1.2.3
# What is the answer?
def compute_the_answer():
    ### BEGIN SOLUTION
    return 42
    ### END SOLUTION

(cell metadata: points=10, ID=1-2-3-test, Autograder tests)

# Do not delete this cell
# The autograder will test the function compute_the_answer
# Visible tests:
assert compute_the_answer() % 1 == 0, 'The answer must be an integer'
assert 0 <= compute_the_answer() <= 100, 'The answer should be between 0 and 100'
# Hidden tests:
### BEGIN HIDDEN TESTS
assert compute_the_answer() == 42, 'The answer is not correct'
### END HIDDEN TESTS

Example release/lab1/lab1.ipynb (generated by nbgrader):

# Question 1.2.3
# What is the answer?
def compute_the_answer():
    # YOUR CODE HERE

# Do not delete this cell
# The autograder will test the function compute_the_answer
# Visible tests:
assert compute_the_answer() % 1 == 0, 'The answer must be an integer'
assert 0 <= compute_the_answer() <= 100, 'The answer should be between 0 and 100'
# Hidden tests:

nbgrader's special cells have an "ID" which this grader (jupyter_grade_server) expects in a specific format. The tests for question number 1.2.3 should have ID 1-2-3a-test, 1-2-3b-test, etc. The extra letter is optional if there is only one test. If the ID is not in this format, the feedback shown to students will not be able to show the correct problem numbers.