Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use on-demand review assignment instead of fixed assignment of reviews (Grab a review) #99

Closed
chrfrantz opened this issue Apr 7, 2019 · 1 comment
Assignees
Labels
enhancement New feature or request improvement Refactoring code or improving UX

Comments

@chrfrantz
Copy link
Collaborator

As discussed; posted here for reference:

Instead of preassigning all reviews to students, why not display the students the number of reviews (not the reviews themselves yet) they have to do and have a "Grab a review" button that students press it when they want to review. Thus, reviews are assigned lazily (random draw amongst least-reviewed submissions) to students and we will always have an open pool of unreviewed (or least-reviewed) submissions. As long as students are signaled how many reviews they have to do, they will grab additional ones (or should - an aspect we can control as discussed below). This enables some opportunities and solves a lot of problems:

  1. Students can review more and less reviews than they have to, i.e. committed students can review more and gain additional marks; students that undercommit lose marks. This would allow us to reward prosocial activity in the class.
  2. If too many students commit, we can have additional review rounds, i.e. some submissions may be reviewed three times if students are keen to review --> increase in review quality
  3. Latecomers can just be injected into the cycles. Any "Grab a review" request would be drawn from the pool of submissions that had the least reviews, i.e., they would be picked first upon the next request - with the exception of own submissions, of course.
  4. We don't have the problem of underreviewing as we have now. Many students may be keen to do additional reviews, but the reviews are associated with students that may not do the reviews (i.e. they are locked) and we will need to deal with that manually.
@JohanAanesen JohanAanesen self-assigned this Apr 23, 2019
@JohanAanesen JohanAanesen added enhancement New feature or request improvement Refactoring code or improving UX labels Apr 23, 2019
@JohanAanesen
Copy link
Owner

This has been implemented as of 2cc2e56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request improvement Refactoring code or improving UX
Projects
None yet
Development

No branches or pull requests

2 participants