Just In Time Homework Assignments

[goehle]

This is the pull request for the just in time assessment and review (jitar) homework type. This is a new adaptive homework type where each homework set is a tree of problems. You can read more about the general description at:

• http://webworkgoehle.blogspot.com/2014/06/just-in-time-adaptive-homework.html
  and more about the various components of this pull request at
• http://webwork.maa.org/wiki/Just-In-Time
• http://webwork.maa.org/wiki/Set_Definition_Files

This is a pretty significant pull request, and will need testing. I'll put testing instructions in a future post, however these changes have been used at a couple of universities for the past year in dozens of courses. So while I'm sure there are issues, the whole system is pretty stable. In addition to the jitar system there are other things that snuck in.

• The jitar homework type, along with all of the small changes needed to support this change.
• Database has been updated to include jitar specific variables. '''Note:''' you will need to perform a database upgrade to use this pull.
• Problems in jitar sets have "tree" addresses. I.e. 1.2.1 is the first subsubproblem of the second subproblem of the first problem. The problem number stored in the database is generated from this tree address using bitmasks, similar to how IP addresses are stored.
• A lot of small changes were made to different files so that problem numbers are translated from their database versions to the correct string. One thing to keep an eye out for is if you see a ridiculously large problem number, its likely a result of not translating a jitar problem number and needs to be fixed.
• There is code added to handle grades and statistics for jitar sets. Students only receive grades for the "top level" problems. However, under certain setups those grades can be the weighted averages of the grades of certain sub problems. See the documentation for more information.
• Added code to control when problems are available to students.
• Added code to send an email alert to a professor if students fail to complete a jitar set.
• In addition to the underlying just in time framework the biggest addition is a replacement for the problem set detail page. This new problem set detail page functions similarly overall with a couple of big exceptions.
  • The problem set detail 2 page does not have drop down problem numbers. Instead problems are reordered by clicking and dragging In general this works well and feels a lot more dynamic.
  • The problem set detail 2 page now renders problems dynamically. Users can render all problems or a single problem, or hide problems after they have been rendered.
  • For jitar sets the page supports collapsible nested lists. This is how the tree structure of the sets is controlled.
• An older addition which got wrapped into this pull request is the student progress bar. This takes the problem list that is usually on the side of the problem page and adds a progress bar to the top and little symbols to indicate if certain problems are done or not.
• Created a new syntax for def files described in wiki link above. The issue is that the "list of values" method of defining problems isn't working out now that chow me another and jitar both add a number of optional parameters. This replaces the problem list portion with a "key value pair" type setup. The old syntax is still supported.
• Some very small fixes:
• Fixed a small issue with newlines and carriage returns and set descriptions.
• Reordered the reduced scoring date field so that if present it is in between the open date and the answer date, i.e. its in chronological order.
• Fixed xml to support rendering knowls hints and solutions
• Allow the problem number to get passed to the xml renderer allowing for server side image cacheing to work.

[mgage]

There is an error in editor3 when trying to add a new version to the homework set. I'm using the version with jitar attached but it's possible this error was there before.

1. Edit a problem (problem1) that you have already in local/......
2. Try to replace the problem with itself (i.e. don't change the name) -- in editor2 and editor3 you get a message saying that local/problem1 already exists.
3. Try appending this problem using editor2 -- you again get the message that the problem exists.
4. Try replacing the problem with itself using editor 3 -- you get the message that the problem exists.
5. Try appending the problem using editor 3 -- you get the message webwork error message: "No display module found for path '/test_jitar/instructor/pgProblemEditor3/Review_1/'. at /opt/webwork/webwork2/lib/WeBWorK.pm line 159."
6. Change the name and try appending problem1a using editor3 you get the webwork error " addGlobalProblem: global problem exists (perhaps you meant to use putGlobalProblem?) at /opt/webwork/webwork2/lib/WeBWorK/ContentGenerator/Instructor.pm line 572."
   7.Try replacing with problem1a and you get the message that problem1a already exists. -- so apparently the file has been written to disk but has not been added to the homeworksets list.
7. If you try to create a stand alone file using either editor2 or editor3 you get the webwork error: "invalid characters in 'problemID' field: 'Undefined_Set' (valid characters are [0-9]) at /opt/webwork/webwork2/lib/WeBWorK/URLPath.pm line 1263."

[mgage]

The editor glitches could be fixed in the next "editor tweaks" pull request. They don't affect jitar very much.

[mgage]

Email bug/feature request for alerting instructor when student fails to successfully complete jitar problem.
This feature can fail silently if the mailer is not working.

Here is what can happen if you try to "email the instructor" when there is more than one instructor in the course.

You get this message: Failed to open the mailer: 5.7.1 - Null sender with multiple recipients not allowed [R0109002] for ""Mike Gage" " on host "smtp-server.rochester.rr.com"

This probably occurs because smtp's are getting stricter about spam and I'm not sure that we properly fill out the "from" field when sending the "email the instructor" message. That's another problem, but at least with the email instructor button there is some alert that the mail is not working.

With the jitar email the instructor the failure is completely silent. At first I thought it was not working at all but after removing the extra instructors so that the smtp server was happy the jitar emailing worked fine.

My short term suggestion is that jitar check that the mailing is successful as "email the instructor" does and send a note to the logfile (not the student) if the mailing fails. (email the instructor should also send a note to the logfile).

A separate project, probably for a group of people at different universities, with different smtp servers, is to overhaul the emailing capability for webwork and make sure it works with smtp servers programmed with varying degrees of strictness about what emails it will accept.

[mgage]

• [+] Comments -- August 30, 2015
  • [+] Student point of view
    • [+] after the due date the student can see all of the
      problems and sub problems. -- expected
    • [+] The icons work very nicely to indicate the kinds of moves
      that can be made
    • [+] attempts =2 -- after two incorrect submissions the open
      subproblems open with yellow highlight -- perfect, as
      expected.
    • [+] if the subproblem opens and you get the main problem
      correct the notice about open subproblems remains. "We
      can live with that"
    • [+] When I come back up to the original problem I can't
      determine which answers I got correct and which ones
      weren't correct without using up another try. The
      answers are sticky, but the answer table has disappeared.
      -- This is as expected, but one should be aware of this
      "feature".
    • [+] You can always try the main problem when the subproblems
      are still open and uncompleted -- I think that is what is
      expected.
    • [+] Took a long time to generate a hardcopy -- and it showed
      all of the sub problems -- but it worked as expected.

[mgage]

• [-] instructor comments -- August 30, 2015
  • [+ ] Tool tips are good.
  • [+ ] Navigation and repositioning of homework problems works
    great!!
  • [+ ] Observation -- you can't move a new problem under a previous
    problem that is collapsed (i.e. already has subproblems but
    the display is collapsed). You have to uncollapse the first
    problem and then move the second problem under it. I think
    this is an excellent feature.
  • [+ ] Check the rendering functionality.
    • [+ ] Works well -- rendering large homework sets can take a
      little time, particularly in images mode if the it's the
      first time the set has been viewed.
  • [+] Reorder problems and add new problems and change problem
    paths.
  • [+] Reorder problems, make sub problems by dragging to the right.
  • [+] Check reordering and how it interacts with sub problems.
  • [+] Check add and delete functionality. Check deleting
    subproblems. Check renumber functionality.
  • [+] Check changing various jitar options, both for the set and
    problems. In particular check that the attempts to open
    children and counts for parent options appear and dissapear
    appropriately.
  • [+] Go to the library browser and add a problem to a jitar set.
  • Use all of the problem editors to add a problem to a jitar
    set, both by append and create new version.
    • See earlier comment/bug report on this feature -- editor3
      needs some work and as does the creation of stand alone
      files
  • [+] View the homework set and problem page for a jitar set. Make
    sure that the various sub problems are formatted correct.
  • [+] Check that the progress bar works, both for jitar sets and
    for regular sets.
  • [+] Check that the progress bar can be turned on and off.
  • [+] Check that the progress bar renders correctly in math 3.
    (Jitar sets are not math 3 compatible).
  • [+] Log in as a student and visit the jitar set. Make sure that
    all of the optoins work in terms of opening and closing sub
    problems and restricting problem progression.
  • [+] Check that the grades page works with jitar sets. For jitar
    problems the grade is the max of the top level problem grade,
    or the weighted average of the child problems which "count
    towards the parent grade".
  • [+] Check that this grade is correctly reported on the homework
    sets page.
  • [+] Check that these grades are correctly reported on the student
    progress and statistics page.
  • [+] Check that the set statistics page has the correct graph
    showing students score on problems and adjusted score on
    problems.
  • [+] Check that the scores are generated correctly in the scoring
    tools page.
  • [+] Check that problem set detail page (version 1) will not try
    to render jitar sets and does not have the jitar option.
  • [+] Check that jitar problem numbers are correctly translated and
    displayed on all pages.
  • [+] Check that jitar hardcopies have the correct problem numbers.
    Note: there isn't really a good solution for things like
    opening children and restricted progression for hardcopies,
    so everything is printed.
  • [+] Export a set and inspect the new def format. (The new def
    format is not supported by homework sets editor 1)
  • [+] Import sets from both the new and the old def formats.
  • Check the email instructor functionality.
    • Check to see that the instructors are sent an alert email.
    • Works -- but see earlier comment about silent fails if
      the smtp isn't accepting emails.
  • [+] Set up a jitar set where all of the problems have a limited
    number of attempts.
  • Fail to correctly answer a problem and all of its sub
    problems.
    • Email is sent to instructor -- but might be silently
      blocked by smtp -- see earlier comment
  • [+] Check that the library browsers will correctly pull problems
    from the new format def files.

[goehle]

The jitar emailer is already set to print debug log messages if there are errors sending failure emails. Is there a different log I should use? As far as I know there isn't an "error" log.

[mgage]

Checked final changes notifying the log about email failures.