Automatic code marking
These scripts will work through a folder of folders containing zipped Java projects and test them using various metrics. The current checks are the following:
- Code compilation.
- Code execution.
- Data output validation (based on random inputs).
- Indentation correctness.
- Variable name quality.
- Code comment quality.
The minimum requirements to run the
batchmark.py script are a task number a path to the folder containing the folders of students' work. The script will then assume default values for everything else.
python batchmark.py <task number> <path to students' work>
python batchmark.py 1 ./DLJ
There are other parameters for controlling where to find inputs and store outputs. Full usage details can be obtained using
python batchmark.py -h at the command line to get the following.
usage: batchmark.py [-h] [-i INITIALS] [-b BUILD] [-t TEMPLATE] [-d DETAILS] [-s SUMMARY] TASK WORK Batch marker for 4001COMP Java programming. positional arguments: TASK Task number (e.g. 1) WORK Folder containing students' folders (e.g. ./DLJ) optional arguments: -h, --help show this help message and exit -i INITIALS, --initials INITIALS Marker's initials (default Master) -b BUILD, --builddir BUILD Folder to output build files to (default ./build) -t TEMPLATE, --template TEMPLATE Word feedback sheeet template (default ./feedback_username.docx) -d DETAILS, --details DETAILS Excel student details list (default ./4001COMP Marking 2014-15.xlsx) -s SUMMARY, --summary SUMMARY Summary of marks as a CSV file (default ./summary.csv)
The automark tool uses python 2.7, along with the following additional requirements:
- Python package xlrd.
- Python package python-docx.
- Python package PLY.
- JDK (javac compiler and VM).
- Python package SOAPpy if (optionally) using Sphere Engine
Scripts for analysing the data generated by the tool, and optimising the parameters used in the scoring calculations, can be found in th
analyse folder. These are for use with GNU Octave. The main process is the
optimise.m script. Most of the other files are function definitions used during the processing.
The main optimisation process assumes various identically-sized one-dimensional matrics have been defined:
comment_orig- The original human-assigned scores for the comments
comment_ave- The calculated average distance between comments
comment_sd- The standard deviation of the distance between comments
indent_orig- The original human-assigned scores for the indentation
indent_error- The number of indentation errors
Each entry represents the result output from checking a single source code submission. The values (apart from the human-assigned scores, obviously) can be extracted from the
summary.csv file output by the automark tool.