E1799 develop #1094
Closed
E1799 develop #1094
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Winbobob
force-pushed
the
master
branch
3 times, most recently
from
2cc51d9
to
a1b78e0
Compare
Winbobob
force-pushed
the
master
branch
9 times, most recently
from
965f660
to
d08c1cb
Compare
Winbobob
force-pushed
the
master
branch
2 times, most recently
from
e813e60
to
3eafd4b
Compare
Winbobob
force-pushed
the
master
branch
2 times, most recently
from
a7726e0
to
4355059
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
E1799. Improve self-review Link peer review & self-review to derive grades
Team Members: Bhavik Patel, Amulya Varote, Arpita Shekhar, Ragavendran Balakrishnan
Mentor: Qaiss Khan Alokozai
What it does: Currently, it is possible to check the “Allow self-review” box on the Review Strategy tab of assignment creation, and then an author will be asked to review his/her own submission in addition to the submissions of others. But as currently implemented, nothing is done with the scores on these self-reviews.
What should be done:
It should be possible to see self-review scores juxtaposed with peer-review scores. Design a way to show them in the regular “View Scores” page and the alternate (heat-map) view. They should be shown amidst the other reviews, but in a way that highlights them as being a different kind of review.
Then implement a way to combine self-review and peer-review scores to derive a composite score. The actual formula is not as important as it is to encapsulate the formula in a method that can easily be modified if we want to change the formula. But the basic idea is that the authors get more points as their self-reviews get closer to the scores given by the peer reviewers. So the function should take as inputs the scores given by peers to a particular rubric criterion and the score given by the author(s). The result of the formula should be displayed in a conspicuous page on the score view.
Of course, there would be no challenge in giving the same self-review scores as the peer reviewers gave if the authors could see peer-review scores before they submitted their self-reviews. So the author(s) should be required to submit their self-evaluation(s) before seeing the results of their peer evaluations.
There are substantial degrees of freedom in this project. The layout of the score display is up to you. You get to choose the formula for computing composite self-/peer-review grades. You get to design an appropriate way to aggregate self-reviews by different members of the project team. Please make these decisions, corresponding with me if you desire, and document them in your design doc, so the instructor and your peer reviewers can give you comments before you proceed to implementation.
Avoid global variables. Allow the instructor to configure weights from the UI. Also … please see papers on the SPARKPlus approach to implementing this and the screenshots from Calibrated Peer Review. These will give you ideas on how other systems approach the same issue.