Computer-aided assessment of mathematics works in the following phases.
- Testing and
- Deploying questions.
- Adding questions to a quiz and use by students.
- Reporting and statistical analysis.
Reviewing students' answers closes the learning cycle for teachers by allowing them to understand what students are doing. Basic reporting is undertaken through the Moodle quiz. For example, lists of scores, etc. are available there. The Moodle quiz also calculates basic statistics based on the numerical information.
To review work in more detail we need to use two important parts of the question. Please read the following two entries before continuing with this article.
Moodle quiz features
Most of the day to day reporting is done through the Moodle quiz features.
For detailed reporting go to
Quiz administration -> Results -> Reporting
The "response" column shows the raw Maxima input value entered by the student and the validity status, e.g. "blank" (not reported), "invalid", "valid", "score". The "response" column also shows the value of the answer note for each potential response tree.
An example note is shown below. In this question a student has answered with
(x-4)^6+c for the input
The potential response tree
prt1 has executed and the note is
ATInt_generic. | 1-0-F.
ans1: (x-4)^6+c [score]; prt1: ATInt_generic. | 1-0-F
- If the potential response tree does not execute it will be reported as
prt1: #to indicate it exists but was not used.
- Reporting is stored in the database, so you may need to "regrade" attempts if you update the answer notes.
- You can download the responses for offline analysis. Split over
;to separate inputs and separate PRT answer notes. Split the individual answer note over
|to get the route through the tree, and the notes added by the answer tests.
Individual STACK item analysis
Please note that the STACK quiz report is still under development. If you have specific requests for features, please contact the developers.
To access the quiz reports you need to navigate to the quiz. Then follow the chain
Administration -> Quiz administration -> Results -> STACK response analysis.
This page lists all the STACK questions in a particular quiz. Click on a link to perform an analysis of attempts at an individual question.
The page contains a number of parts.
- A summary of the question. This includes the question variables, un-instantiated (i.e. raw) question text, general feedback, and the un-instantiated question note as a reminder to the teacher what the question is about.
- Summary tables of the frequencies of answer notes for each potential response tree, and each variant.
- A detailed summary of all attempts at a particular variant.
- Maxima code to facilitate off-line analysis.
The first table provides a summary of the frequency of each answer note for each question note, both as a numerical frequency and as a percentage. Each potential response tree is listed separately. A second table shows similar data for splits answer notes, to show the frequency each branch of the tree is traversed. Depending on the form of the trees in the question, each table provides potentially useful comparisons of the frequencies of outcomes of different variants.
For each question variant we provide a number of tables of data.
- The first table lists all inputs at one question, their frequency, the score generated and each outcome from the potential response trees.
- Next we produce tables for each input, and count the number of times each expression is entered and its validity. Where an input is invalid a note records the reason. Remember, validity is more than syntactic soundness so a syntactically-valid expression might be "invalid" for this input. For example, we might forbid floating-point numbers.
We create a number of lists in Maxima code which record all the valid inputs. These can be copied into a desktop version of Maxima for off-line analysis. This reduces server load, but is also more flexible enabling teachers to manipulate answers without having to update the server. In order to use the analysis you will need to set up the STACK - Maxima sandbox.
The code generates three Maxima lists
- the list
inputscreates the names of the inputs to the question. These are assumed to be valid Maxima variable names, so are Maxima objects. E.g.
- the list
variantswhich is a list of strings which are the question notes used.
- the list of lists called
stackdata. This holds all the input data. The elements of the list show the input data for each variant. E.g.
stackdatais the input data for
variant. For each variant, the elements are a list of all data for each corresponding input. E.g.
stackdatais the input data for
This data is available for off line analysis using functions in the file
More work on this functionality is needed.