Send [weekly] reminder emails to evaluators prior to deadline

[gabestein]

Motivations/Context

Update Unjournal email reminders for accepted evaluators to align with “bonus” (3 week) and “final” (five week) deadlines

Requirements

• Change email sending behavior and templates to match sending six emails as described in "proposed new behavior"

BEHAVIOR SPEC

Current behavior

1. Set deadline 21 days (3 weeks) from the ‘Accept’ current date
2. Send one reminder emails, if evaluator has not submitted evaluation

• 14 days from ‘Accept’ (7 days before deadline): If evaluator has not submitted evaluation, send accepted evaluator reminder email

Proposed new behavior

• Deadline is set in the Evaluation Manager integration configuration (#202)
• Note: Make sure that deadline cannot be set to fewer than 21 days
• Schedule six reminder emails, all relative to the deadline

1. deadline - 21 days: If evaluator has not submitted evaluation, send accepted evaluator reminder email 1
2. deadline - 14 days: If evaluator has not submitted evaluation, send accepted evaluator reminder email 2
3. deadline - 7 days: If evaluator has not submitted evaluation, send accepted evaluator reminder email 3
4. deadline - 1 day: If evaluator has not submitted evaluation, send accepted evaluator reminder email 4
5. deadline + 6 days: If evaluator has not submitted evaluation, send accepted evaluator reminder email 5
6. deadline + 8 days: If evaluator has not submitted evaluation, send accepted evaluator reminder email 6

REMINDER EMAILS TEXT

Reminder Emails

1. Accepted Evaluator Reminder Email 1 [MODIFY EXISTING]

Send (deadline - 21 days)

Italics preceded by DELETE indicates text to delete

Bold text is text to add

Subject: [Unjournal] Reminder to evaluate “{{paper-title}}” by next week for prompt evaluation bonus

Hi {{invited-evaluator-first-and-last-name}},

Thanks again for agreeing to evaluate “{{paper-title}}” for The Unjournal. [DELETE: This note is a reminder that your evaluation should be submitted by {{one-week-before-deadline}}]. This note is a reminder to submit your evaluation by {{deadline-minus-14-days-date}} to receive a $100 “prompt evaluation bonus,” in addition to your baseline compensation. Please note that after {{deadline-date}} we will consider re-assigning the evaluation, and later submissions may not be eligible for the full baseline compensation.

Please submit your evaluation and rating, as well as any specific considerations, using this evaluation form. The form includes instructions and information about the paper**/project**.

If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

Once your evaluation has been submitted and reviewed, we will follow up with details about payment and next steps.

Thanks and best wishes,

{{em-name}}

Unjournal.org

2. Accepted Evaluator Reminder Email 2 [NEW]

Send (deadline - 14 days)

Subject: [Unjournal] Reminder: Submit evaluation by {{21-days-from-accept-date}} for prompt evaluation bonus

Hi {{invited-evaluator-first-and-last-name}},

Thank you again for agreeing to evaluate “{{paper-title}}” for The Unjournal. This note is a reminder to submit your evaluation by {{deadline-minus-14-days-date}} to receive a $100 “prompt evaluation bonus,” in addition to your baseline compensation. Please note that after {{deadline-date}} we will consider re-assigning the evaluation, and later submissions may not be eligible for the full baseline compensation.

Please submit your evaluation and rating, as well as any specific considerations, using this evaluation form. The form includes instructions and information about the paper.

If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

Once your evaluation has been submitted and reviewed, we will follow up with details about payment and next steps.

Thanks and best wishes,

{{em-name}}

Unjournal.org

3. Accepted Evaluator Reminder Email 3 [NEW]

Send (deadline - 7 days)

Subject: [Unjournal] Reminder to evaluate “{{paper-title}}” by next week

Hi {{invited-evaluator-first-and-last-name}},

Thank you again for agreeing to evaluate “{{paper-title}}” for The Unjournal. This note is a reminder that your evaluation should be submitted by {{deadline-date}} (next week). Please note that after that date we will consider re-assigning the evaluation, and later submissions may not be eligible for the full baseline compensation.

Please submit your evaluation and rating, as well as any specific considerations, using this evaluation form. The form includes instructions and information about the paper.

If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

Once your evaluation has been submitted and reviewed, we will follow up with details about payment and next steps.

Thanks and best wishes,

{{em-name}}

Unjournal.org

4. Accepted Evaluator Reminder Email 4 [NEW]

send (deadline - 1 day)

Subject: [Unjournal] Reminder: evaluation due tomorrow

Hi {{invited-evaluator-first-and-last-name}},

Thank you again for agreeing to evaluate “{{paper-title}}” for The Unjournal. This note is a reminder that your evaluation should be submitted by {{deadline-date}}. Please note that after tomorrow we will consider re-assigning the evaluation, and later submissions may not be eligible for the full baseline compensation.

Please submit your evaluation and rating, as well as any specific considerations, using this evaluation form. The form includes instructions and information about the paper.

If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

Once your evaluation has been submitted and reviewed, we will follow up with details about payment and next steps.

Thanks and best wishes,

{{em-name}}

Unjournal.org

5. Accepted Evaluator Reminder Email 5 [NEW]

send (deadline + 6 days)

Subject: [Unjournal] Reminder: evaluation overdue, to be reassigned

Hi {{invited-evaluator-first-and-last-name}},

Thank you again for agreeing to evaluate “{{paper-title}}” for The Unjournal. This note is a reminder that your evaluation is overdue. As previously noted, we are now planning to reassign the evaluation to another evaluator.

If you have completed the evaluation but forgot to submit it, please submit your evaluation and rating today, as well as any specific considerations, using this evaluation form. The form includes instructions and information about the paper. (And in this case, we will follow up with details about payment and next steps.)

If we don’t hear from you by the end of {{7-days-after-deadline-date}}, we will remove you from this assignment and you will no longer be eligible for compensation.

If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

Thanks and best wishes,

{{em-name}}

Unjournal.org

6. Accepted Evaluator Reminder Email 6 [NEW]

send (deadline + 8 days)

Subject: [Unjournal] removed from evaluation assignment—reassigning

Hi {{invited-evaluator-first-and-last-name}},

This is just a quick note to let you know that we have removed you from the evaluation assignment (for the paper/project) “{{paper-title}}.” We are reassigning this to another evaluator. Please let us know if you believe we have made a mistake here. If you have any questions, do not hesitate to reach out to me at {{em-email-address}}.

We are sorry that you were not able to complete the evaluation in time, but we understand you are busy and these things happen.

Best wishes,

{{em-name}}

Unjournal.org

Acceptance Criteria

• Final deadline constant configurable in instance configuration
• Submission deadline and email send dates are computed from date of acceptance + instance deadline configuration
• Email content should be hard-coded
• Unschedule reminder emails on evaluation submission (via job key)

Extra Work

• Users editing emails