Permalink
Browse files

item analysis report contributed by Enrique Castro

  • Loading branch information...
1 parent 339c1d1 commit 0fc5939983046b8c00a49a1ed42381da195619f6 gustav_delius committed May 15, 2005
@@ -0,0 +1,17 @@
+<p align="center"><b>Analysis Options</b></p>
+
+<p>The range of quiz attempts that are procesed and analyzed can be selected.
+There are two parameters to specify which attempts are used: </p>
+
+<p><b>Attempt selection:</b></p>
+It may be convenient to analyze just one quiz attempt for each user.
+This particular attempt may be the one with the highest overall score,
+the first attempt or the last attempt of those performed. Or all attempts data may
+be collected for a cummulative analysis.</p>
+
+<p><b>Rejection of low scores:</b></p>
+<p>Sometimes the users are just browsing the quiz, exploring it,
+and not going through it for a grade. It is common that such 'trial' attempts
+get very low scores. These byassing data can be excluden from analysis by
+setting a low limit for the score of the attempts to analyze. This limit is
+specified as a percentage (0-100) of the maximum grade achievable in the quiz.</p>
@@ -0,0 +1,61 @@
+<p align="center"><b>Item Anaysis</b></p>
+
+<p>This table presents processed quiz data in a way suitable for anayzing and judging the
+performance of each question for the function of assessment. The statistical parameters used are
+calculated as explained by classical test theory (ref. 1)</p>
+
+<p><b>Facility Index (% Correct)</b></p>
+<p>This is a measure of how easy or difficult is a question for quiz-takers.
+It is calculated as:
+<br>
+FI = (X<sub>average</sub>) / X<sub>max</sub>
+<br>
+
+where X<sub>average</sub> is the mean credit obtained by all users attempting the item, <br>
+and X<sub>max</sub> is the maximum credit achievable for that item.<br>
+If questions can be distributed dicotomically into correct / uncorrect categories,
+this parameter coincides with the percentage of users that answer the question correctly. </p>
+
+</p>
+<p><b>Standard Deviation (SD)</b></p>
+<p>This parameter measures the spread of answers in the response population. If all users
+answers the same, then SD=0. SD is calculated as the statistical stadard deviation for the
+sample of fractional scores (achieved/maximum) at each particular question.
+</p>
+
+<p><b>Discrimination Index (DI)</b></p>
+<p>This provides a rough indicator of the performance of each item to separate proficient
+<i>vs.</i> less-proficient users. This parameter is calculated by first dividing learners into thirds
+based on the overall score in the quiz. Then the average score at the analyzed item is calculated for
+the groups of top and bottom performers, and the average scored substracted. The matematical expression is:
+<br/>
+DI = (X<sub>top</sub> - X<sub>bottom</sub>)/ N
+<br/>
+
+where X<sub>top</sub> is the sum of the fractional credit (achieved/maximum) obtained at this item by the 1/3 of users having tha highest
+grades in the whole quiz (i.e. number of correct responses in this group), <br/>
+and X<sub>bottom</sub>) is the analog sum for users with the lower 1/3 grades for the whole quiz.</p>
+
+<p>This parameter can take values between +1 and -1.
+If the index goes below 0.0 it means that more of the weaker learners got the item right than the stronger learners.
+Such items should be discarded as worthless. In fact, they reduce the accuracy of the overall score for the quiz. </p>
+
+<p><b>Discrimination Coefficient (DC)</b></p>
+<p>This is another measure of the separating power of the item to distinguish proficient from weak learners.<br/>
+The discrimination coefficient is a correlation coefficient between scores at the item and at the whole quiz. Here it is calculated as:
+
+<br/>
+DC = Sum(xy)/ (N * s<sub>x</sub> * s<sub>y</sub>)
+<br/>
+where Sun(xy) is the sum of the products of deviations for item scores and overall quiz scores, <br>
+N is the number of responses given to this question<br/>
+s<sub>x</sub> is the standard deviation of fractional scores for this question and, <br/>
+s<sub>y</sub> is the standard deviation of scores at the quiz as a whole.
+</p>
+<p>Again, this parameter can take values between +1 and -1. Positive values indicate items that
+do discriminate proficient learners, whereas negative indices mark items that are answered best by
+those with lowest grades. Items with negative DC are answered incorrectly by the seasoned learners
+and thus they are actually a penalty against the most proficient learners. Those items should be avoided.<br/>
+The advantage of Discrimination Coefficient vs. Discrimitation Index is that the former uses information
+from the whole population of learners, not just the extreme upper and lower thirds. Thus, this parameter may be
+more sensitive to detect item performance.</p>
@@ -77,6 +77,7 @@
$string['penaltyscheme'] = 'Apply penalties';
$string['previewquiz'] = 'Preview quiz';
$string['previous'] = 'Previous state';
+$string['questiontype'] = 'Question type $a';
$string['readytosend'] = 'You are about to finalize this quiz attempt. Are you sure you want to continue?';
$string['rendering'] = 'Rendering';
$string['renderingserver'] = 'Rendering Server';
@@ -115,4 +116,10 @@
$string['withselected'] = 'With selected';
$string['wronguse'] = 'You can not use this page like that';
+$string['analysisoptions'] = 'Analysis options';
+$string['attemptselection'] = 'Select which attempts to analyze per user ';
+$string['lowmarkslimit'] = " Don't analyze if score is less than: ";
+$string['reportanalysis'] = 'Item analysis';
+$string['allattempts'] = 'all attempts';
+$string['analysistitle'] = 'Item Analysis Table';
?>
View
@@ -400,6 +400,71 @@ function get_correct_responses(&$question, &$state) {
}
/**
+ * Return an array of values with the texts for all possible responses stored
+ * for the question
+ *
+ * All answers are found and their text values isolated
+ * @return array An array of values giving the responses corresponding
+ * to all answers to the question. Answer ids are used as keys
+ * @param object $question The question for which the answers are to
+ * be retrieved. Question type specific information is
+ * available.
+ */
+ // ULPGC ecastro
+ function get_all_responses($question, $state) {
+ unset($answers);
+ if (is_array($question->options->answers)) {
+ foreach ($question->options->answers as $aid=>$answer) {
+ unset ($r);
+ $r->answer = $answer->answer;
+ $r->credit = $answer->fraction;
+ $answers[$aid] = $r;
+ }
+ } else {
+ $answers[]="error"; // just for debugging, eliminate
+ }
+ $result->id = $question->id;
+ $result->responses = $answers;
+ return $result;
+ }
+
+ /**
+ * Return the actual response to the question in a given attempt
+ * for the question
+ *
+ * @return compond object fractiongrade & actual response .
+ *
+ * @param object $question The question for which the correct answer is to
+ * be retrieved. Question type specific information is
+ * available.
+ * @param object $state The state object that corresponds to the question,
+ * for which a correct answer is needed. Question
+ * type specific information is included.
+ */
+ // ULPGC ecastro
+ function get_actual_response($question, $state) {
+ /* The default implementation only returns the raw ->responses.
+ may be overridden by each type*/
+ //unset($resp);
+ if (isset($state->responses)) {
+ return $state->responses;
+ } else {
+ return null;
+ }
+ }
+
+ // ULPGC ecastro
+ function get_fractional_grade($question, $state) {
+ $maxgrade = $question->maxgrade;
+ $grade = $state->grade;
+ if ($maxgrade) {
+ return (float)($grade/$maxgrade);
+ } else {
+ return (float)$grade;
+ }
+ }
+
+ /**
* Prints the question including the number, grading details, content,
* feedback and interactions
*
@@ -1711,25 +1776,44 @@ function quiz_print_correctanswer($text) {
* @param object $question The question object for which the icon is required
* @param boolean $editlink If true then the icon is a link to the question
* edit page.
+* @param boolean $return If true the functions returns the link as a string
*/
-function quiz_print_question_icon($question, $editlink=true) {
-// Prints a question icon
+function quiz_print_question_icon($question, $editlink=true, $return = false) {
+// returns a question icon
global $QUIZ_QUESTION_TYPE;
global $QUIZ_QTYPES;
+
+ $html = '<img border="0" height="16" width="16" src="questiontypes/'.
+ $QUIZ_QTYPES[$question->qtype]->name().'/icon.gif" alt="'.
+ get_string($QUIZ_QTYPES[$question->qtype]->name(), 'quiz').'" />';
if ($editlink) {
- echo "<a href=\"question.php?id=$question->id\" title=\""
- .$QUIZ_QTYPES[$question->qtype]->name()."\">";
+ $html = "<a href=\"question.php?id=$question->id\" title=\""
+ .$QUIZ_QTYPES[$question->qtype]->name()."\">".
+ $html."</a>\n";
}
- echo '<img border="0" height="16" width="16" src="questiontypes/';
- echo $QUIZ_QTYPES[$question->qtype]->name().'/icon.gif" alt="';
- echo get_string($QUIZ_QTYPES[$question->qtype]->name(), 'quiz').'" />';
- if ($editlink) {
- echo "</a>\n";
+ if ($return) {
+ return $html;
+ } else {
+ echo $html;
}
}
+// ULPGc ecastro
+function quiz_get_question_review($quiz, $question) {
+// returns a question icon
+ $qnum = $question->id;
+ $context = $quiz->id ? '&amp;contextquiz='.$quiz->id : '';
+ $quiz_id = $quiz->id ? '&amp;quizid=' . $quiz->id : '';
+ return "<a title=\"$strpreview\" href=\"javascript:void();\" onClick=\"openpopup('/mod/quiz/preview.php?id=$qnum$quiz_id','$strpreview','scrollbars=yes,resizable=yes,width=700,height=480', false)\">
+ <img src=\"../../pix/t/preview.gif\" border=\"0\" alt=\"$strpreview\" /></a>";
+
+}
+
+
+
+
/**
* Print the question image if there is one
*
@@ -1892,6 +1976,46 @@ function quiz_print_quiz_question(&$question, &$state, $number, $quiz, $options=
}
/**
+* Gets all teacher stored answers for a given question
+*
+* Simply calls the question type specific get_all_responses() method.
+*/
+// ULPGC ecastro
+function quiz_get_question_responses($question, $state) {
+ global $QUIZ_QTYPES;
+ $r = $QUIZ_QTYPES[$question->qtype]->get_all_responses($question, $state);
+ return $r;
+}
+
+
+/**
+* Gets the response given by the user in a particular attempt
+*
+* Simply calls the question type specific get_actual_response() method.
+*/
+// ULPGC ecastro
+function quiz_get_question_actual_response($question, $state) {
+ global $QUIZ_QTYPES;
+
+ $r = $QUIZ_QTYPES[$question->qtype]->get_actual_response($question, $state);
+ return $r;
+}
+
+/**
+* Gets the response given by the user in a particular attempt
+*
+* Simply calls the question type specific get_actual_response() method.
+*/
+// ULPGc ecastro
+function quiz_get_question_fraction_grade($question, $state) {
+ global $QUIZ_QTYPES;
+
+ $r = $QUIZ_QTYPES[$question->qtype]->get_fractional_grade($question, $state);
+ return $r;
+}
+
+
+/**
* Gets the default category in a course
*
* It returns the first category with no parent category. If no categories
@@ -291,6 +291,41 @@ function grade_responses(&$question, &$state, $quiz) {
return true;
}
+
+ // ULPGC ecastro for stats report
+ function get_all_responses($question, $state) {
+ unset($answers);
+ if (is_array($question->options->subquestions)) {
+ foreach ($question->options->subquestions as $aid=>$answer) {
+ unset ($r);
+ $r->answer = $answer->questiontext." : ".$answer->answertext;
+ $r->credit = 1;
+ $answers[$aid] = $r;
+ }
+ } else {
+ $answers[]="error"; // just for debugging, eliminate
+ }
+ $result->id = $question->id;
+ $result->responses = $answers;
+ return $result;
+ }
+
+ // ULPGC ecastro
+ function get_actual_response($question, $state) {
+ unset($results);
+ if (isset($state->responses)) {
+ foreach($state->responses as $left=>$right){
+ $lpair = $question->options->subquestions[$left]->questiontext;
+ $rpair = $question->options->subquestions[$right]->answertext;
+ $results[$left] = $lpair." : ".$rpair;
+ }
+ return $results;
+ } else {
+ return null;
+ }
+ }
+
+
}
//// END OF CLASS ////
@@ -371,6 +371,28 @@ function grade_responses(&$question, &$state, $quiz) {
return true;
}
+ // ULPGC ecastro
+ function get_actual_response($question, $state) {
+ $answers = $question->options->answers;
+ /*
+ if (!$question->options->single) {
+ echo "Objeto State-responses: <br>tamaño= ".count($state->responses)."<br> ";
+ print_object($state->responses);
+ foreach ($state->responses as $resp){
+ echo "esto es response id= ".$rid." <br>";
+ print_object($resp);
+ echo " Tamaño interno=".count($resp);
+ foreach ($resp as $r){
+ print_object($r);
+ }
+ }
+ }*/
+ foreach ($state->responses as $rid){
+ $answer = $answers[$rid]->answer;
+ $responses[] = $answer;
+ }
+ return $responses;
+ }
}
//// END OF CLASS ////
@@ -163,7 +163,24 @@ function get_correct_responses(&$question, &$state) {
->get_correct_responses($wrappedquestion, $state);
}
- function print_question(&$question, &$state, $number, $quiz, $options) {
+ // ULPGC ecastro
+ function get_all_responses($question, $state){
+ global $QUIZ_QTYPES;
+ $wrappedquestion = &$state->options->question;
+ return $QUIZ_QTYPES[$wrappedquestion->qtype]
+ ->get_all_responses($wrappedquestion, $state);
+ }
+
+ // ULPGC ecastro
+ function get_actual_response($question, $state){
+ global $QUIZ_QTYPES;
+ $wrappedquestion = &$state->options->question;
+ return $QUIZ_QTYPES[$wrappedquestion->qtype]
+ ->get_actual_response($wrappedquestion, $state);
+ }
+
+
+ function print_question(&$question, &$state, &$number, $quiz, $options) {
global $QUIZ_QTYPES;
$wrappedquestion = &$state->options->question;
$QUIZ_QTYPES[$wrappedquestion->qtype]
Oops, something went wrong. Retry.

0 comments on commit 0fc5939

Please sign in to comment.