Gateway Answer Bug#740
Conversation
… you run out of attempts, and the answer and due dates are not equal.
|
Students shouldn’t see correct answers or solutions until after the answer date. It looks like your patch gives them access to solutions as soon as they run out of attempts. Doesn’t this defeat the purpose of the answer date? Maybe Gavin can chime in on this, since I believe he wrote this code, but to me it looks like the line you deleted and the two above it should all go. Danny This fixes a couple of bugs. The first is that sometimes the "hide_score_by_problem" variable for old sets which has been imported is undefined. So we want to assume that that is a "N" if its not there. The second issue is that on gateways where the answer date differed from the due date, solutions and correct answers weren't shown even if the student had run out of attempts. This was intentionally added by someone but doesn't make sense to me. If someone has a reason why it should be this way let me know. To test:
You can view, comment on, or merge this pull request online at: Commit Summary
File Changes
Patch Links:
— |
|
In principle you are right. However because of the way the gateway system works if you have a gateway with a time limit each version of the gateway will get the appropriate close date but will share the answer date of the parent. So there isn't really a way to show solutions on individual timed gateways. People at Western want this functionality because they want to have "practice" exams which allow students to see solutions to problems they got wrong. If you wanted students to see the score after they finish the test, but not the solutions, you should set "Show problems on finished tests" to "After the Answer Date". This would show scores after the test is finished but students couldn't see the problems (hence the solutions and the correct answers) until after the actual answer date. Thats not a perfect solution either, I'll admit. |
|
I looked into this and found I was wrong above. If the Gateway quiz is timed then each version of the quiz gets its own due date and its own answer date. However, there is no indication on the Problem Sets page whether or not answers are available for a given test. I see two possibilities for how this should work.
I'm not sure which option is better. Right now we have a hybrid where you can get either behaviour, and its not obvious which one you will get with a given combination of settings. |
|
This may have actually been what was intended: that scenario 1 occurs when you set the answer date the same as the due date, and scenario 2 occurs when you don’t, but you’re right; it’s definitely not obvious how to obtain either. Is there a good way to implement this so that both behaviours are possible and are easily understood? Does adding another setting make sense? On Sep 2, 2016, at 12:56 PM, Geoff Goehle <notifications@github.commailto:notifications@github.com> wrote: I looked into this and found I was wrong above. If the Gateway quiz is timed then each version of the quiz gets its own due date and its own answer date. However, there is no indication on the Problem Sets page whether or not answers are available for a given test. I see two possibilities for how this should work.
I'm not sure which option is better. Right now we have a hybrid where you can get either behaviour, and its not obvious which one you will get with a given combination of settings. — |
|
Assuming I understand the issue... What if we allowed answer dates to come earlier than due dates? In such a situation, you still don't see answers unless you are past the answer date and all attempts are used up. The easy way to get scenario 1 is to set answer date = open date. |
|
The original thinking on the gateway test module was that it would be used for asynchronous tests that students would take (multiple times) before the deadline. To accomplish this, each time a student takes a test they get a new entry in the user_set table, which is the new set version. In this case the two most likely possibilities are that on would want the student to see the test results on completing the set version, or that (similar to a homework assignment) there should be a delay of some sort to encourage reflection. The original treatment of the open/answer dates accomplishes this. In this model setting the answer date to be after the global set due date isn't likely to be needed. The "show answers" and "show scores" options were intended as providing a mechanism to add restrictions to this behavior, as noted. Without speaking to the underlying data or code, I'm inclined to think that the best way to address this is probably to have the open/answer date input as usual, and to reframe the "show scores" options as something like
This could be a rewrite of the underlying parameters stored in the user_set table, or could just change how data are set from the form input. For what it's worth, |
|
Hi Everyone, I'm a newbie so please excuse me if I am missing something, but it looks like you marked issue number 1 above, "sometimes the "hide_score_by_problem" variable for old sets which has been imported is undefined. So we want to assume that that is a "N" if its not there", as fixed back in September of 2016? But we just upgraded to version 2.13 a few months ago and are now experiencing this exact problem with our gateway quizzes.... I found some solutions on the forum, here: http://webwork.maa.org/moodle/mod/forum/discuss.php?d=3976#p12737 |
|
I've pulled PR #852 which should fix the problem with "hide_score_by_problem" not being imported |
|
Short answer: I don't think there is a consensus on the desired behavior of the answer date for gatewayquiz assignments. In the following I use "global" to describe the global set; "user" the set assigned to the user; and "version" to describe one of the set versions that is created for a user when s/he starts work on the user set. The difficulty in deciding on what the open/due/answer dates should do stems, I think, from there being two different meanings for the open/due dates for these assignments. The first is the open/due date for gatewayquiz that can be taken many times in a time interval. The second is the open/due date for a version of the assignment. These could be essentially the same (e.g., if a the gatewayquiz is given in class), or can be radically different (e.g., if it's a skills test that the student can take many times in a multiple week interval). In particular: for the user set, the open date is the date when students can start taking versions of the test. The due date is the date students are no longer allowed to start taking versions of the test. For a version of the set, the open date is the date (and time) when a student begins work on the test (which will be when the "take test" link is clicked), and the due date is the date (and time) by which the test must be completed (which will be the open date + the version time limit, which is determined by the user set). The answer date for the version set should be the date (and time) at which the student can see the answers, as it is for a normal set. However, it is not clear (to me) how to generate that logically from the global or user set data. For those who use the gatewayquiz assignments as skills tests that are taken many times by students over a time interval restricting to the behavior that answers only become available on tests after the answer date of the user set (that is, versions of the test would always have the same answer date as the user set and answers would not become available if a student runs out of attempts) will not work. (This is Geoff's option 2.) As a side note, the hide work and hide score parameters were added to accomodate situations in which one doesn't want the test details widely disseminated. We use these for our placement test. They function differently from answer availability, allowing access (or restricting access) to a different set of data. Gavin |
|
Thanks Gavin. That helps. |
mgage
added
the
Needs UI decision consensus
|
When WW 3 is developed - this needs to be examined and design/implementation decisions need to be made. |
When a quiz version is created, its answer date is set to be equal to the close date plus the difference between the templates close date and answer date. So when configuring if and when scores and answers are available, this can depend on either the template answer date or the version answer date. Which date was used was inconsistent, plus there was some additional logic for when the answer date was the same as the due date. This adds a new option to hide_work and hide_score, 'BeforeVersionAnswerDate', so it can be configured which answer date to use. Removes the condition of if the answer date and due date are equal. And makes use of the template or version answer date consistent with the quiz settings. Discussion about some of this behavior can be found in openwebwork#740.
When a quiz version is created, its answer date is set to be equal to the close date plus the difference between the templates close date and answer date. So when configuring if and when scores and answers are available, this can depend on either the template answer date or the version answer date. Which date was used was inconsistent, plus there was some additional logic for when the answer date was the same as the due date. This adds a new option to hide_work and hide_score, 'BeforeVersionAnswerDate', so it can be configured which answer date to use. Removes the condition of if the answer date and due date are equal. And makes use of the template or version answer date consistent with the quiz settings. Discussion about some of this behavior can be found in openwebwork#740.
When a quiz version is created, its answer date is set to be equal to the close date plus the difference between the templates close date and answer date. So when configuring if and when scores and answers are available, this can depend on either the template answer date or the version answer date. Which date was used was inconsistent, plus there was some additional logic for when the answer date was the same as the due date. This adds a new option to hide_work and hide_score, 'BeforeVersionAnswerDate', so it can be configured which answer date to use. Removes the condition of if the answer date and due date are equal. And makes use of the template or version answer date consistent with the quiz settings. Discussion about some of this behavior can be found in openwebwork#740.
When a quiz version is created, its answer date is set to be equal to the close date plus the difference between the templates close date and answer date. So when configuring if and when scores and answers are available, this can depend on either the template answer date or the version answer date. Which date was used was inconsistent, plus there was some additional logic for when the answer date was the same as the due date. This adds a new option to hide_work and hide_score, 'BeforeVersionAnswerDate', so it can be configured which answer date to use. Removes the condition of if the answer date and due date are equal. And makes use of the template or version answer date consistent with the quiz settings. Discussion about some of this behavior can be found in openwebwork#740.
7 participants
This fixes a couple of bugs. The first is that sometimes the "hide_score_by_problem" variable for old sets which has been imported is undefined. So we want to assume that that is a "N" if its not there.
The second issue is that on gateways where the answer date differed from the due date, solutions and correct answers weren't shown even if the student had run out of attempts. This was intentionally added by someone but doesn't make sense to me. If someone has a reason why it should be this way let me know. To test: