CS111 autograder and test cases
Autograding scripts for Wellesley's CS111 (Introductory CS) course. The program evaluates student Python code on a suite of test cases, and generates HTML outputs with error hints, as well as json logs. It supports grading for a variety of functions, including those with print statements and some support for graphics.
Currently in active development. The codebase is adapted from the Berkeley AI autograder.
How to create a new problem set skeleton
Clone this repo and pull for latest changes before starting work.
Create a new directory for the problem set somewhere on your machine. Let's refer to it as
inspector/to this new directory:
$ cp otterInspect.py /path/to/ps0x $ cp honorcode.py /path/to/ps0x $ cp testCaseReader.py /path/to/ps0x $ cp casefile_creator.txt /path/to/ps0x $ cp -R inspector /path/to/ps0x ```
Change to the the problem set directory
$ cd /path/to/ps0x/ ```
Delete the sample
inspector/pickledcontents (but leave the directories).
$ rm -r inspector/test_cases/* inspector/picked/* ```
casefile_creator.txtwith your desired test cases, descriptions, and grade weights. For complex arguments or results, create a pickle file with a
.pickleextension, drop it in the
pickled/directory, and put the filename in
casefile_creator.txt. If you pickle arguments, the entire argument list should be pickled as a tuple.
Note that for functions with print side-effects, you must create an object of type
inspector/testClasses.py), pickle it, and set the "print" column in
For functions with randomness, set the seed to 0 before you generate your test cases.
$ python testCaseReader.py
This populates the
inspector/test_casesdirectory with files readable by the otterInspect.
You can now delete
casefile_creator.txtsince the students don't need these.
inspector/urls.txtwhere each line has a task id (like "task1") and the URL to the problem set description of the task.
Edit the first few lines of the
otterInspect.pyby specifying the filenames for the student code, project name, and due date.
Place the solution files for the problem set, and all supporting files in your
/path/to/ps0xdirectory. Test that running
works as expected.
*.pycfiles and the files generated by your testing runs of otterInspect:
$ rm *.pyc $ rm your_result.html $ rm -r .logs ```
- Scrub the solutions to create skeleton files. Voila! The problem set is ready for release.