Periodic randomization

[abasyrov]

This is a replacement of pull request #615 with corrections and modifications suggested by Geoff.
Feel free to close #615 without merging

Slightly revised comments from #615 are below.

• the feature could be turned on/off on the course level
• the course-level default period for re-randomization could be set; the current default is 5 attempts
• per-problem setting of re-randomization period is also possible
• the per-problem re-randomization period is exported to set-def files (in new format) regardless of the course-wide setting
• the re-randomization period value is also imported from set-def file in new format regardless of the course-wide settings (if it is present in the set-def file); otherwise the $problemDefaults{prPeriod} is used
• if the problem set is imported from an old-format set-def file, the per-problem re-randomization period is set to the value of $problemDefaults{prPeriod}

[mgage]

I'm including some clarifying discussions from the earlier pull request so that they don't get lost:

Thanks for your contribution; this seems like an interesting feature. It allows professors to have something which is kind of half way between unlimited attempts and limited attempts. You should know this is a bit of an awkward time because we are trying to get release/2.11 ready to be spun on off and there is a reasonably large pull request for "just-in-time" type homework sets #580 which has a non-zero intersection with this. That pull request is going to be merged into develop first, so what is likely going to happen is that you will need to either wait until #580 is pulled into develop (or rebase this branch off of #580) and make some changes to this code. The big ones I see so far are:

The changes you made to ProblemSetDetail (and fixes for the issues below) will need to be side ported to ProblemSetDetail2.
The changes you made to importing/exporting set defs will need to be redone to fit with the changes to ProblemSetList2 in Just-In-Time. In particular right now there are errors if you import a file created before this patch with the patched code (e.g. SetDemo) and the notation that is used in the export (with the :) is too hacky. This isn't really your fault. The old def format wasn't set up well for problems to have a lot of potentially optional parameters. There is a new format in the Just-In-Time branch for specifying problems which is better for this kind of thing (see http://webwork.maa.org/wiki/Set_Definition_Files ). Basically you shouldn't mess with the old problemList parser in ProblemSetList2 and just add your parameter keys to the problemListV2 parser so that it will recognize them.
As I said, those changes will have to wait until #580 is added to develop, or you will have to do some fancy git rebasing. There are some other issues which can be looked at right now:

Your options in defaults.config should be put into localOverrides.conf.dist. The local overrides file is where site administrators will set site wide defaults.
The "Rerandomize after" box shows up on the Problem Set Detail page, even if the option is turned off. To keep the interface as simple as possible all text/options/boxes/whatever related to a feature should be turned off if the feature isn't enabled.
The default number of attempts between re-randomization is really high. It doesn't have to be changed, but according to the data gathered so far in the big data project average number of attempts total on any given problem is less than 5.
The parameter value of 0 for Rerandomize after does not show up as "Never" in the box. This is probably a bug in line 606 of ProblemSetDetail.pm where some or all of those or's should be defined or's.
One issue, and it may or may not be something you want to deal with, is that a new seed does not guarantee a new problem. Most problems only have a handful of possible values for their random parameters, and there are quite a few problems with no random parameters at all. The ShowMeAnother feature has some code which tries to find a seed which gives a different problem text. Its possible you might want to use that.
I would be interested in hearing why you have the "Request New Version" button rather than automatically creating a new version and putting a message on the Problem page saying that you have done so. There will some more testing and code review in the future. However this should be enough to work with for now. Thanks again for your submission to the project.
@abasyrov

abasyrov commented 29 days ago
I would be interested in hearing why you have the "Request New Version" button rather than automatically creating a new version and putting a message on the Problem page saying that you have done so.
If we go with automatic creation of new version, then we would be subjecting students to the following confusing scenario at the last attempt before re-randomization:

student submits their answer to the old version
webwork grades the answers against the old version (or maybe new version -- it depends on exact placement of code) of the problem
webwork feedback is shown above the new version of the problem.
With the 'request new version' button option, the student would submit their answer to the old version, it would be graded against that same version of the problem, and webwork feedback would be shown above the old version of the problem and the 'request new version' button would be shown. After the 'request new version' button is pressed, the new version is generated and presented to the student.

I'll work on the rest of the issues you've pointed out. I thought about guaranteeing a new version of the problem (if possible) -- I think that the code from 'show me another' could be easily stolen/adapted to re-randomization. However, that would only guarantee that the new version is different from the old one, but it does not prevent the 'version cycle'. If the problem has only two versions in total the student would see alternating sequence of the two versions.
@goehle
Owner
goehle commented 29 days ago
The "Request New Version" thing makes sense.

The version cycle is not something you can prevent, I think. Even more of
an issue are problems with no randomization at all. But again, you cant do
anything about that.
…

[goehle]

Sorry its taken so long to get back to you on this. I only have a few small things and then it can be pulled

• You need a period after "will be requested" https://github.com/openwebwork/webwork2/pull/626/files#diff-a07041547ae3443045c3ba8201e0e807R1726
• For problem set detail 2 could you add a ? icon with a tool tip describing the field, similar to "Show me another". All you need to do is add a "help_text" parameter like https://github.com/openwebwork/webwork2/blob/develop/lib/WeBWorK/ContentGenerator/Instructor/ProblemSetDetail2.pm#L368

Thanks for your patience and your contribution.

[abasyrov]

I've made the suggested changes in commit e9dcd7d, abasyrov@e9dcd7d

I was not sure if github emailed about that.

[goehle]

Hi abasyrov, github doesn't email when commits are updated, only when comments are made. Thanks for letting me know.

[goehle]

After testing this seems like a good addition and I'm going to merge the pull request.

Thanks for your contribution abasyrov. I'm certain that there will be issues/tweaks/whatever that come up as it is distributed more wildly. If and when those come up well use this thread to contact you.