diff --git a/Trifold Huge.png b/Trifold Huge.png
new file mode 100644
index 0000000..19cabbd
Binary files /dev/null and b/Trifold Huge.png differ
diff --git a/Trifold Medium.png b/Trifold Medium.png
new file mode 100644
index 0000000..f8ab6a6
Binary files /dev/null and b/Trifold Medium.png differ
diff --git a/Trifold.png b/Trifold.png
index 1bd1350..a5f56ab 100644
Binary files a/Trifold.png and b/Trifold.png differ
diff --git a/Trifold.svg b/Trifold.svg
index edcc44e..3a1ebff 100644
--- a/Trifold.svg
+++ b/Trifold.svg
@@ -7,6 +7,7 @@
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
+ xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="4320"
@@ -15,11 +16,41 @@
version="1.1"
inkscape:version="0.48.4 r9939"
sodipodi:docname="Trifold.svg"
- inkscape:export-filename="C:\Users\William\Documents\GitHub\primes\Trifold.png"
- inkscape:export-xdpi="30"
- inkscape:export-ydpi="30">
+ inkscape:export-filename="C:\Users\William\Documents\GitHub\primes\Trifold Medium.png"
+ inkscape:export-xdpi="90"
+ inkscape:export-ydpi="90">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ inkscape:snap-nodes="false"
+ inkscape:snap-bbox-midpoints="false"
+ inkscape:snap-to-guides="true"
+ inkscape:snap-object-midpoints="true"
+ inkscape:snap-center="true">
image/svg+xml
-
+
@@ -402,16 +475,2178 @@
inkscape:groupmode="layer"
id="layer1"
transform="translate(0,2187.6378)">
+
+
+ style="overflow:hidden"
+ id="g3359"
+ transform="matrix(2.0179373,0,0,2.0179373,1170,235.21422)">
+
+
+
+ Figure 2b:
+
+ Max error vs Calibrated error
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 0
+
+
+ 20
+
+
+ 40
+
+
+ 60
+
+
+ 80
+
+
+ 0
+
+
+ 20
+
+
+ 40
+
+
+ 60
+
+
+ 80
+
+
+
+
+
+ Absolute error in grading Calibrated set
+
+
+ Max absolute error in grading
+
+
+
+
+
+
+
+
+
+ Figure 2a:
+ Uncalibrated Error vs Reward
+
+ d
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 0
+
+
+ 0.05
+
+
+ 0.1
+
+
+ 0.15
+
+
+ 0.2
+
+
+ 0.25
+
+
+ 0.3
+
+
+ -20
+
+
+ 0
+
+
+ 20
+
+
+ 40
+
+
+ 60
+
+
+
+
+
+ Reward
+
+
+ Error in grading Uncalibrated set
+
+
+
+
+
+
Workload / Objective Function vs Various Mechanisms Teaching a Class to Grade Itself Using Game TheoryTeaching a Class to Grade Itself Using Game TheoryWilliam Wu and Nicholaas Kaashoek William Wu and Nicholaas KaashoekActon Boxborough Regional High School and Lexington High School Calibration Mechanism Mechanisms Massive Open Online Courses (MOOCs) are a great way to learn for free! However, with so many students enrolled in these courses, the large number of assignments can be hard to grade. Currently, two main tactics are being used:In Massive Open Online Courses (MOOCs), the large number of assignments can be hard to grade. Currently, two main tactics are being used:Computer grading - Computers are experts at grading multiple-choice, but not so much for essays or open-response.Computer grading - Computers are experts at multiple-choice, but have yet to conquer essays.Peer grading - Current peer grading systems tend to have loopholes that clever students can use to their advantage.Peer grading - Current peer grading systems tend to have loopholes that clever students can use.We propose several peer grading systems that, according to game theory, should create accurate grades, reduce overall grading work, and be loophole-free in the long term. These systems are based on our student model - a set of assumptions we believe students abide by. We also created a quantitative benchmark to compare between our and existing mechanisms. We propose several peer grading mechanisms based on game theory and our student model - a set of assumptions we believe students abide by. We also create a benchmark to compare between our and existing mechanisms.
+ transform="translate(-1.5673448,304.07303)">
Student recieves two assignments Student receives two assignments
+ style="color:#000000;fill:#93ffc6;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:2.12132049;marker:none;visibility:visible;display:inline;overflow:visible;enable-background:accumulate"
+ transform="translate(0,54.23334)">
Take off no points on one paper0 units of effort 1 unit of effort on one paper0 units of effort
Earn H(G) Earn H(0) Partner takes more pts and gets refutedEarn H(G)+2
+ inkscape:connector-curvature="0"
+ sodipodi:nodetypes="cc" />
Deduction Mechanism Introduction Introduction
+ transform="translate(-0.6738987,-219.88455)">
Student recieves two assignments Student receives two assignments
+ style="fill:#93ffc6;fill-opacity:1;stroke:none"
+ transform="translate(0,54.233328)">
0 units of work
+ style="fill:#92ff98;fill-opacity:1;stroke:none"
+ transform="translate(0,27.116664)">
Recieve 0Receive 0Recieve full creditReceive full creditEarn H(G)
@@ -2128,7 +4339,7 @@
xml:space="preserve"
id="flowRoot3579-3"
style="font-size:40px;font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Arial;-inkscape-font-specification:Arial"
- transform="translate(2781.0092,-39.876446)">Benchmark Model & Assumptions Model & Assumptions In order to measure exactly which mechanisms are better than others, we created a numerical benchmark where a lower score is better. The score is computed by adding the most work done by any person to the highest possible error in grading. Mathematically:To compare mechanisms, we created a numerical benchmark (objective function) where a lower score is better. The score is computed by adding the most work done by any person to the highest possible error in grading. Mathematically:i} } In the short term, our assumptions could be made more realistic, as we consider issues such as how to generate accurate grades from incompetent graders as well as students only partially grading assignments. With more realistic assumptions, more realistic mechanisms are also required.As we consider how to generate accurate grades from incompetent graders, our model will require more realistic assumptions, which in turn may create more complex mechanisms.In the long term, we would like to see our model used by others to create new mechanisms, or existing mechanisms implemented in MOOCs such as Edx or Coursera. Eventually, we would like to see new mechanisms based off of our model or our existing mechanisms implemented in MOOCs such as EdX or Coursera. We began by creating a student model consistng of assumptions predicting how students would realistically act. Using a croudsourced experiment, we validated key assumptions and a system of incentivization. The model is less than realistic, and can still be further improved.An effective and accurate solution is required to grade the massive number of assignments created in MOOCs.We began by creating a student model - a set of assumptions that approximate the realistic behavior of students. We then developed various grading mechanisms based on our student model and mechanism design. These mechanisms would incentivize students to grade accurately and efficiently as proven by game theory.Based off our model, we developed various grading mechanisms using mechanism design that, according to our student model and game theory, would incentivize students to grade accurately and efficiently. These mechanisms optimally perform in a less realistic student model and can still work reasonably well in a smaller classroom.We tested our Calibration mechanism using a croudsourced experiment, finding that it could work in practice. A mechanism based on a more realistic model would achieve better results.Our model can easily be reused and improved upon by future researchers who wish to develop more efficient solutions. Efficiency is measured in terms of the benchmark we created, a numerical score encompassing both the accuracy of grades and the effort spent by any one person. We hope that our work lays a steady foundation upon which improvements can be made. Our model can easily be reused and improved upon by future researchers who wish to develop more efficient solutions. Efficiency is measured in terms of the benchmark we created, a numerical score encompassing both the accuracy of grades and the effort spent by any one person. To the best of our knowledge, this is the first game-theory-based peer grading system. Analysis Analysis This mechanism achieves a benchmark score of 4, which is very low, as the greatest deviation is 2, and the greatest amount of work done by any one person is also 2. 2+2=4.This mechanism is extremely simple, and relies on the unrealistic assumption that students are unable to communicate with each other. If they were able to, students would be able to figure out which paper is calibrated by checking which one they all share. We tested and verified the Calibration mechanism through an anonymous croudsourced experiment. In this case, "grading" involved counting two sets of colored objects. Initially unknown to the participant, one set is "calibrated" and will be used to reward the participant based on the accuracy of the grading. This mechanism achieves a benchmark score of 2, which is very low, as the greatest deviation is 0, and the greatest amount of work done by any one person is 2. 2+0=2.This mechanism is far more complex than the last one created, but is actually more efficient. In a classroom environment where the professor has very high control over the class, and all the students are very capable graders as the mechanism does not account for incompetent graders The Deduction mechanism achieves a very low benchmark score of 2, consisting of a 2 in max work done and a 0 in max error in grade. Incapable graders will raise the benchmark score, as they issue refutations that add work to the professor. [1] Tony Bates. What’s right and what’s wrong about coursera-style moocs. Online Learn- ing and Distance Eductation Resources, 2012.[2] Zack Budryk. Dangerous curves. Inside Higher Ed, 2013.[3] Sir John Daniel. Making sense of moocs: Musings in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, 2012.[4] Anne Herrington and Charles Moran. Writing to a machine is not writing at all. National Writing Project, 2012.[5] Noam Nisan, Tim Roughgarden, E. Tardos, and V. V. Vazirani, editors. Algorithmic Game Theory. Cambridge University Press, 2007.[6] Laddie Odom. A swot analysis of the potential impact of moocs. EdITLib, 2013.[7] Eric Randall. Edx now has software to grade your essays. Boston Magazine, 2013.[8] Ruth S. Can moocs and existing e-learning paradigms help reduce college costs? Inter- national Journal of Technology in Teaching and Learning, pages 21–32, 2012.[9] Theophrastus. The problems with moocs 1: Robo essay-grading. BLT - Bible*Literature*Translation, 2013.[10] Audrey Watters. The problems with peer grading in coursera. Inside Higher Ed, 2012. [1] Tony Bates. What's right and what's wrong about Coursera-style MOOCs. Online Learning and Distance Education Resources, 2012.[2] Noam Nisan, Tim Roughgarden, E. Tardos, and V. V. Vazirani, editors. Algorithmic Game Theory. Cambridge University Press, 2007.[3] Laddie Odom. A swot analysis of the potential impact of MOOCs. EdITLib, 2013.[4] Eric Randall. EdX now has software to grade your essays. Boston Magazine, 2013.[5] Audrey Watters. The problems with peer grading in Coursera. Inside Higher Ed, 2012. References
+ id="flowPara3780">References Results Overall, our Calibration and Deduction mechanisms vastly outperform existing solutions (see Figure 4) with the exception of Improved Calibration. The Calibration mechanism is described visually in Figure 1. It achieves a low benchmark score of 4, of a 2 in max work done and 2 in max error in grade.The Calibration mechanism can break when students communicate to reveal the calibrated assignment to circumvent grading. The Improved Calibration mechanism mitigates this issue by introducing multiple calibrated papers at the expense of more work, raising the objective score. Acknowledgments We would like to thank MIT and the MIT PRIMES program for providing the research opportunity as well as our parents and mentors for their support. Figure 1: The Calibration Mechanism Figure 3: The Deduction Mechanism The Calibration mechanism rewards students for accurately grading the Calibrated set in order to reduce the grading error for the uncalibrated set, as shown in Figure 2a. This is also evident in Figure 2b, where less error in grading the Calibrated set correlates to a smaller maximum grading error. An accurate grade for the uncalibrated set is what the Calibration mechanism tries to achieve, and the data suggests that the Calibration mechanism works in practice. R2 = 0.853 Figure 4: Workload/Objective Function vs Various Mechanisms