Merge pull request #394 from edx/pmitros/author-hints

Add pedagogical guidance in B&R guide

en_us/shared/building_and_running_chapters/building_course/establish_grading_policy.rst

@@ -123,6 +123,14 @@ assignment types.
 To create a new assignment type, in the bottom of the Grading page, select
 **New Assignment Type**, then configure the fields described below.
 
+.. note:: In a MOOC, some number of learners will experience personal 
+  emergencies, technical problems (such as power blackouts or loss of 
+  internet access), or other issues. You will not be able to grant 
+  individual extensions to such learners, but can provide some amount 
+  of flexibility for managing such issues by designing longer deadlines 
+  and allowing droppable assignments.
+
+
 ==========================
 Assignment Type Fields
 ==========================
@@ -164,6 +172,7 @@ You configure the following fields for each assignment type.
   The number of assignments of this type that the grader will drop. The grader
   will drop the lowest-scored assignments first.
 
+
 .. _Graded Subsections:
 
 **********************************************

en_us/shared/building_and_running_chapters/building_course/export_import_course.rst

@@ -108,8 +108,11 @@ To import a course:
 Work with the .tar.gz file
 ******************************
 
-Courses are exported and imported in .tar.gz files.  
-
+Courses are exported and imported in .tar.gz files containing OLX, an edX
+format for open course exchange. For more information about OLX, see the `edX
+OLX Documentation <http://edx.readthedocs.org/projects/edx-open-learning-
+xml/en/latest/>`_.
+
 To extract and compress .tar.gz files, you will need a third-party program.
 
 If you are using Microsoft Windows, see the following resources.

en_us/shared/building_and_running_chapters/building_course/pages.rst

@@ -94,7 +94,8 @@ Hide or Show the Course Wiki Page
 ************************************************
 
 By default, your course includes a wiki page. Learners and course staff can
-use the wiki to post content and comment on others' content.
+use the wiki to post content as well as comment on other people's content. For
+details, see :ref:`Course_Wiki`.
 
 If you do not want to use a wiki in your course, you can hide the page.
 
@@ -116,6 +117,7 @@ Select it again to make the **Wiki** page visible.
  page, and any students logged in to edX can access the content if they know
  the URL.
 
+
 .. _Reorder Pages:
 
 ****************

en_us/shared/building_and_running_chapters/building_course/setting_up_student_view.rst

@@ -34,6 +34,12 @@ based on the content of the page.
 
 You configure the contents of this page in Studio, as described in this topic.
 
+.. note:: Given the diversity of MOOC learners, be careful to clearly 
+   communicate the  target audience, level, and prerequisites for your course. 
+   Aim for concrete, unambiguous words (for example, ``understand eigenvalue 
+   decomposition'' rather than ``intermediate linear algebra'').
+
+
 .. _The Learner Dashboard:
 
 ***********************************

en_us/shared/building_and_running_chapters/creating_content/create_problem.rst

@@ -67,7 +67,7 @@ All problems on the edX platform have several component parts.
    above) and the explanation (see 10 below). The instructor sets whether
    the **Show Answer** button is visible.
 
-#. **Attempts.** The instructor may set a specific number of attempts or allow
+#. **Attempts.** The instructor might set a specific number of attempts or allow
    unlimited attempts for a problem. By default, the course-wide **Maximum
    Attempts** advanced setting is null, meaning that the maximum number of
    attempts for problems is unlimited. If the course-wide **Maximum Attempts**
@@ -304,15 +304,22 @@ Maximum Attempts
 ==============================
 
 This setting specifies the number of times a student is allowed to attempt
-answering the problem. By default, the course-wide **Maximum Attempts**
+answering a problem. By default, the course-wide **Maximum Attempts**
 advanced setting is null, meaning that the maximum number of attempts for
 problems is unlimited. If the course-wide **Maximum Attempts** setting is
 changed to a specific number, the **Maximum Attempts** setting for individual
 problems defaults to that number, and cannot be set to unlimited.
 
-.. note:: Only questions that have a **Maximum Attempts** setting of 1 or 
- higher are included in the answer distribution computations used in edX
- Insights and the Student Answer Distribution report.
+.. note:: edX recommends setting **Maximum Attempts** to either unlimited or a
+   very large number. Unlimited attempts allow for mastery learning and
+   encourages risk taking and experimentation, both of which lead to improved
+   learning outcomes. Allowing for unlimited attempts might be impossible in
+   some courses, such as those where grading is primarily based on multiple
+   choice questions.
+
+.. note:: Only questions that have a **Maximum Attempts** setting of 1 or
+   higher are included in the answer distribution computations used in edX
+   Insights and the Student Answer Distribution report.
 
 
 .. _Problem Weight:
@@ -440,7 +447,12 @@ This setting has the following options.
 |                   | other students see.                  |
 +-------------------+--------------------------------------+
 
-.. note:: The edX Platform has a 20-seed limit for randomization.
+.. note:: The edX Platform has a 20-seed limit for randomization. This means 
+  that in a course with a large numbers of learners, those learners will see 
+  a maximum of 20 different problem variants. This limitation allows for 
+  better analysis of learner submissions (such as the ability to detect 
+  common incorrect answers and usage patterns around such answers).
+
 
 .. _Show Answer:
 

en_us/shared/building_and_running_chapters/creating_content/create_video.rst

@@ -4,9 +4,23 @@
 Working with Video Components
 #############################
 
-You can create videos and add them to a course to supplement other components,
-such as discussions and problems, to promote active learning. Adding a video
-to your course has several steps.
+You can create videos and add them to a course to supplement active
+learning components, such as discussions and problems. Videos can be
+effective for a number of purposes, such as presenting motivating 
+material, showing experiments, and reducing cognitive load for complex
+content.
+
+.. note:: Before creating video content, figure out whether video is
+  the best medium. Should the content be conveyed through text? Through
+  an interactive demonstration? By having the student work through a 
+  problem? Try to keep videos as short as possible -- students are much
+  less likely to finish watching a video if it is more than 5-10 minutes 
+  long. When creating videos, try to follow 
+  `Richard Mayer's 12 Principles <http://hartford.edu/academics/faculty/fcld/data/documentation/technology/presentation/powerpoint/12_principles_multimedia.pdf>`_. 
+  These principles are based on extensive experimental research of student
+  learning. 
+
+Adding a video to your course has several steps.
 
 .. contents:: Section Contents 
   :local:

en_us/shared/building_and_running_chapters/running_course/course_wiki.rst

@@ -8,9 +8,11 @@ Wikis provide a way for course staff and students to access, share, and
 collaboratively edit information both about, and for, your course. 
 
 Every course has a wiki that you can set up in a way appropriate for your
-course. You can influence how students (and staff) use it by :ref:`seeding the
-wiki<Seeding the Wiki>` with specific content, by explaining how you want it to
-be used at the beginning of the course, and by providing clear instructions and
+course. If you have specific expectations for how the wiki should be
+used, communicate these expectations to your students and staff. You can
+:ref:`seed the wiki<Seeding the Wiki>` with specific content and provide a
+skeleton structure and some exemplars. At the beginning of the course, explain
+how you want the course wiki to be used, and provide clear instructions and
 guidelines for its use.
 
 Common uses for the course wiki might include the following activities.

en_us/shared/exercises_tools/checkbox.rst

@@ -17,6 +17,12 @@ For the checkbox problems in your course, you can use edX Insights to review
 aggregated learner performance data and examine submitted answers. For more
 information, see `Using edX Insights`_.
 
+.. note:: On checkbox problems, make sure the all of the choices are
+   unambiguous, and avoid trick questions. Checkbox problems with ambiguity
+   can be frustrating to students, especially if they have a limited number of
+   attempts.
+
+
 ****************************
 Creating a Checkbox Problem
 ****************************

en_us/shared/exercises_tools/mult_choice_num_input.rst

@@ -9,7 +9,7 @@ You can create a problem that combines a multiple choice and numerical input pro
 .. image:: ../../../shared/building_and_running_chapters/Images/MultipleChoice_NumericalInput.png
   :alt: Image of a multiple choice and numerical input problem
 
-.. note:: Currently, students can only enter numerals in the text field. Students cannot enter words or mathematical expressions.
+.. note:: Currently, students can only enter numerals in the text field. Students cannot enter words or mathematical expressions, which might be confusing to students used to normal edX numerical inputs.
 
 .. _Create an MCNI Problem:
 
@@ -42,4 +42,4 @@ Multiple Choice and Numerical Input Problem Code
   <choice correct="true">False. The correct value is <numtolerance_input answer="3.14"/>.</choice>
   </radiotextgroup>
   </choicetextresponse>
-  </problem>
+  </problem>

en_us/shared/exercises_tools/multiple_choice.rst

@@ -31,6 +31,46 @@ However, you cannot switch back to the Simple Editor from the Advanced Editor.
 Therefore, you might want to format the problem as completely as possible
 before you begin to use the Advanced Editor.
 
+********************************************************
+Pedagogical Considerations for Multiple Choice Questions
+********************************************************
+
+edX recommends the use, whenever possible, of authentic assessments 
+rather than multiple choice questions for graded problems. Authentic
+assessments in online courses tend to lead to better learning
+outcomes. In addition, authentic assessments allow for infinite
+attempts, mastery learning, and more intellectual risk taking, both
+of which lead to substantially better learning outcomes.
+
+Multiple choice questions do have several helpful uses:
+
+* Ungraded multiple choice questions can help students think about a concept 
+   in the context of knowledge transfer.
+* For many subject areas, authentic assessments are either unavailable
+   or prohibitively complex to use. In such courses, multiple choice
+   questions can act as the only available fallback.
+
+Fortunately, multiple choice questions are among the best studied in
+assessment literature. A few guidelines for the creation of such
+questions:
+
+* Organize the set of answers logically. Use consistent phrasing
+   between answers, and when possible, parallel structure. 
+* Place as many of the words in the stem, and keep the answers
+   as concise as possible.
+* The distractors should not be substantially shorter, longer,
+   or use different structure than the answer. The answers should
+   be as consistent in structure, length, and phrasing as possible.
+* Avoid negatives (and especially double negatives)
+* Test higher-order thinking (comprehension and critical thinking). 
+   Avoid simple recall.
+* If there is a finite number of attempts, avoid trick questions
+   and try to keep wording clear and unambiguous. 
+* Make all distractors plausible.
+* Use "All of the above" and "None of the above" with caution. If a 
+   learner can identify at least two correct answers, it can give away
+   the answer with only partial comprehension. 
+
 .. _Use the Simple Editor to Create a Multiple Choice Problem:
 
 ================================================================

en_us/shared/subsections/open_response_assessments/OpenResponseAssessments.rst

@@ -198,9 +198,12 @@ points possible)
      - Salic law
      - This response explains the way that Salic law contributed to the dynastic disagreement between Edward III and Philip VI, leading to the Hundred Years' War.
 
-For more information about writing effective rubrics, see Heidi Goodrich
-Andrade's `Understanding Rubrics
-<http://learnweb.harvard.edu/alps/thinking/docs/rubricar.htm>`_.
+.. note:: For peer grading, the most effective rubrics are as concrete 
+   and specific as possible. Many novice learners will be unqualified 
+   to make the types of value judgments required for more holistic
+   rubrics. In addition, edX suggests using clear, simple language in 
+   rubrics.
+
 
 For more information, see :ref:`PA Add Rubric`.