Skip to content

Commit

Permalink
Update README wrt no function expressions, and graderPrograms array
Browse files Browse the repository at this point in the history
  • Loading branch information
ning-y committed Jul 13, 2018
1 parent 185c596 commit 8f7b35f
Showing 1 changed file with 46 additions and 18 deletions.
64 changes: 46 additions & 18 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,25 @@ Grader

**The grader is still in development, API specficiations may change!**

The grader is a component of the `Cadet backend`_. The grader:
The grader is a component of the `Cadet backend`_. The grader,

1. Receives two strings, a student program and, a grader program
2. Concatenates them end-to-end
3. Evaluates the single combined program in the `js-slang`_ interpreter
4. Returns a ``GraderOutput`` containing the result of the evaluation
1. Receives a student program string, and an array of grader program strings
2. For each grader program,

1. Concatenate the student program and grader program into a single combined program
2. Evaluates the single combined program in the `js-slang`_ interpreter

3. Returns a ``GraderOutput[]`` containing the result of the evaluation

.. _Cadet backend: https://github.com/source-academy/cadet
.. _js-slang: https://github.com/source-academy/js-slang

Grader Program
==============

The grader program is a program written in `the source language`_. In the Grader, the grader program will be concatenated with the student program. This means that,
The grader programs are programs written in `the source language`_. In the Grader, the grader programs will be individually concatenated with the student program. This means that,

- The grader program will have access to the global scope of the student program,
- Each grader program will have access to the global scope of the student program,
- and that there is no restrictions to how the grader program is written,
- **save that the grader program returns a number.** [1]_

Expand All @@ -46,14 +49,16 @@ and the grader program [2]_,

.. code-block:: javascript
(function() {
function uovaej1pheeVai7ohkoh() {
const sum_marks = sum(111, 234) === 345 ? 1 : 0;
const mul_marks = mul(11, 9) === 99 ? 1 : 0;
const pow_marks = pow(2, 8) === 256 ? 2 : 0;
return sum_marks + mul_marks + pow_marks;
})();
}
uovaej1pheeVai7ohkoh();
The grader's output for this function is then,
The grader's corresponding output for this function is then,

.. code-block:: json
Expand All @@ -71,10 +76,12 @@ All ``SourceErrors`` are fatal, i.e. execution stopping. Therefore, it may be ad

.. code-block:: javascript
(function() {
function uovaej1pheeVai7ohkoh() {
const pow_marks_edge = pow(2, 0) === 0 ? 1 : 0;
return pow_marks_edge;
})();
}
uovaej1pheeVai7ohkoh();
Of course, this is an infinite loop. The source interpreter raises a runtime error [3]_,

Expand Down Expand Up @@ -124,24 +131,45 @@ For example, the above gradings may be represented like so,
</SOLUTION>
</SNIPPET>
<GRADER>
(function() {
function uovaej1pheeVai7ohkoh() {
const sum_marks = sum(111, 234) === 345 ? 1 : 0;
const mul_marks = mul(11, 9) === 99 ? 1 : 0;
const pow_marks = pow(2, 8) === 256 ? 2 : 0;
return sum_marks + mul_marks + pow_marks
})()
return sum_marks + mul_marks + pow_marks;
}
uovaej1pheeVai7ohkoh();
</GRADER>
<GRADER>
(function() {
function uovaej1pheeVai7ohkoh() {
const pow_marks_edge = pow(2, 0) === 0 ? 1 : 0;
return pow_marks_edge;
})();
}
uovaej1pheeVai7ohkoh();
</GRADER>
</PROBLEM>
</PROBLEMS>
The grader then receives a two-element array of strings, each corresponding to the content of one of the above ``GRADER`` nodes. The overall output of the grader is then (using the student program near the top of this document),

.. code-block:: json
[
{
"resultType": "pass",
"marks": 4
},
{
"resultType": "pass",
"marks": 0
}
]
Recall that a ``resultType`` of ``"pass"`` only means that no SourceErrors were raised in the evaluation. The order of ``GraderOutput`` elements in the array are preserved according to the order of ``GRADER`` nodes in the assessment XML files.

Note that the ``SOLUTION`` node is not related to the Grader, but a node used by a previous iteration of the source academy.

.. [1] In fact, the grader program accepts any return value from the combined student and grader programs; but the `Cadet backend`_ expects only a number, to be entered into the database.
.. [2] While staff have the flexibility to design the grader program in whatever style they fancy, it is recommended to nest *everything* in a function expression to avoid problems with variable scoping.
.. [2] While staff have the flexibility to design the grader program in whatever style they fancy, it is recommended to nest *everything* in a function to avoid problems with variable scoping. Thus, only one identifier is created. Name this function such that it avoids possible collisions with the identifiers that the student may create, e.g. with randomly generated alphanumeric strings.
.. [3] Only if the execution is fast enough to exceed the maximum stack before the service times out. Either way, no marks are awarded for the grading. Syntax errors, even in the grader program, will also result in a return ``mark`` of 0.

0 comments on commit 8f7b35f

Please sign in to comment.