Merge pull request #755 from edx/carol/doc/CA_ORAStaffGrading

DOC-2444: Course author doc for ORA staff grading

en_us/course_authors/source/exercises_tools/open_response_assessments/Manage_ORA_Assignment.rst

@@ -0,0 +1 @@
+.. include:: ../../../../shared/subsections/open_response_assessments/Manage_ORA_Assignment.rst

en_us/course_authors/source/exercises_tools/open_response_assessments/index.rst

@@ -10,4 +10,5 @@ Open Response Assessments
 
    OpenResponseAssessments
    CreateORAAssignment
-   Access_ORA_Info
+   Manage_ORA_Assignment
+   Access_ORA_Info

en_us/olx/source/components/open_response_assessments/Manage_ORA_Assignment.rst

@@ -0,0 +1 @@
+.. include:: ../../../../shared/subsections/open_response_assessments/Manage_ORA_Assignment.rst

en_us/olx/source/components/open_response_assessments/index.rst

@@ -13,5 +13,6 @@ Open Response Assessments
 
    OpenResponseAssessments
    CreateORAAssignment
+   Manage_ORA_Assignment
    Access_ORA_Info
 

en_us/open_edx_course_authors/source/exercises_tools/open_response_assessments/Manage_ORA_Assignment.rst

@@ -0,0 +1 @@
+.. include:: ../../../../shared/subsections/open_response_assessments/Manage_ORA_Assignment.rst

en_us/open_edx_course_authors/source/exercises_tools/open_response_assessments/index.rst

@@ -13,4 +13,5 @@ Open Response Assessments
 
    OpenResponseAssessments
    CreateORAAssignment
-   Access_ORA_Info
+   Manage_ORA_Assignment
+   Access_ORA_Info

en_us/shared/subsections/open_response_assessments/Access_ORA_Info.rst

@@ -1,218 +1,56 @@
 .. _Accessing ORA Assignment Information:
 
-##########################################
-Accessing Assignment and Learner Metrics
-##########################################
+######################################
+Accessing Metrics for ORA Assignments
+######################################
 
 After you release an open response assessment assignment, you can access
-information about the number of learners in each step of the assignment or the
-performance of individual learners. This information is available in the
-**Course Staff Information** section at the end of each assignment. To access
-learner information, open the assignment in the courseware, scroll to the
-bottom of the assignment, and then select the black **Course Staff
-Information** banner.
-
-.. image:: ../../../../shared/images/PA_CourseStaffInfo_Collapsed.png
-   :alt: The Course Staff Information banner at the bottom of the peer
-    assessment.
-
-When you access a specific learner's information for an open response
-assessment, you can view his responses and uploaded files. If necessary, you
-can :ref:`cancel the learner's submission<Remove a learner response from peer
-grading>` so that it is not forwarded to peers for assessment.
+various metrics for the assignment. For example, you can view the number of
+learners in each step of the assignment or in possible states such as
+"Waiting" or "Completed" within the assignment. In addition to viewing metrics
+for the assignment, you can also access assignment details for an individual
+learner.
 
-.. _PA View Metrics for Individual Steps:
-
-************************************************
-View Metrics for Individual Steps
-************************************************
-
-You can check the number of learners who have completed, or are currently
-working through, the following steps:
-
-* Submitted responses.
-* Completed peer assessments.
-* Waiting to assess responses or receive grades.
-* Completed self assessments.
-* Completed the entire assignment.
-
-To find this information, open the assignment in the courseware, scroll to the
-bottom of the assignment, and then select **Staff Info**.
-
-The **Staff Info** section expands, and you can see the number of learners who
-are currently working through (but have not completed) each step of the
-problem.
-
-.. image:: ../../../../shared/images/PA_CourseStaffInfo_Expanded.png
-   :alt: The Course Staff Information box expanded, showing problem status.
-
-.. _Access Information for a Specific Learner:
-
-***********************************************
-Access Information for a Specific Learner
-***********************************************
-
-You can access the following information about an individual learner's
-performance on a peer assessment assignment.
-
-* The learner's response.
-* Any files that the learner uploaded.
-* The peer assessments that other learners performed on the learner's
-  response, including feedback on individual criteria and on the overall
-  response.
-* The peer assessments that the learner performed on other learners'
-  responses, including feedback on individual criteria and on the overall
-  responses.
-* The learner's self assessment.
-
-In the following example, you can see the performance information for a
-specific learner. This learner's response received one peer assessment, and the
-learner completed a peer assessment on one other learner's response. The
-learner also completed a self assessment.
-
-.. image:: ../../../../shared/images/PA_SpecificStudent.png
-   :width: 500
-   :alt: Report showing information about a learner's response.
-
-To determine whether this learner has received the required number of
-assessments from other learners and has completed the required number of
-assessments for other learners, refer to the **Graded By** and **Must Grade**
-values that were set for the open response assessment assignment in Studio. For
-more information about these settings, see :ref:`Specify Step Settings<PA
-Specify Step Settings>`.
-
-For details about accessing information for a specific learner, and for an
-example that shows a learner's response with more assessments, see :ref:`Access
-Learner Information`.
-
-
-.. _Access Learner Information:
-
-=======================================
-Access a Specific Learner's Information
-=======================================
-
-Before you can access information about a specific learner's assignment, you
-need the learner's username. For more information, see :ref:`View and download
-student data`.
-
-#. In the LMS, go to the peer assessment assignment that you want to access.
-
-#. Scroll to the bottom of the problem, and then select **Course Staff
-   Information**.
+For information about tasks that you can perform on learner responses in an
+ORA assignment, including :ref:`performing a grade override
+assessment<Override a learner assessment grade>` or :ref:`cancelling a
+learner's submission<Remove a learner response from peer grading>`, see
+:ref:`Managing ORA Assignments`.
 
-#. In the **Get Student Info** box, enter the learner's username, and then
-   select **Submit**.
 
-The learner's response appears along with additional available information. If
-the learner uploaded a file along with her response, you can select **View the
-file associated with this submission** to review or download it.
-
-The example that follows includes this information for an assessment.
-
-* The learner's response.
-* The two peer assessments for the response.
-* The two peer assessments that the learner completed.
-* The learner's self assessment.
-
-For a larger view, select the image so that it opens by itself in the browser
-window, and then click anywhere on the image that opens.
-
-.. image:: ../../../../shared/images/PA_SpecificStudent_long.png
-   :width: 250
-   :alt: Report showing information about a learner's response.
-
-
-.. _Remove a learner response from peer grading:
+.. _PA View Metrics for Individual Steps:
 
 ************************************************
-Remove a Learner's Response from Peer Grading
+View ORA Assignment Statistics
 ************************************************
 
-If you use open response assessments, learners might alert you to vulgar,
-abusive, or otherwise inappropriate responses that they have seen while
-performing peer assessments. In such a situation you can :ref:`locate<Locate a
-specific ORA submission>` and cancel the submission. Doing so removes the
-inappropriate response from peer assessments so that it is no longer shown to
-other learners.
-
-.. note:: Removing a learner's submission is an irreversible action.
-
-When you cancel an inappropriate submission, the response is immediately
-removed from the pool of submissions available for peer assessment. If the
-inappropriate response has already been sent to other learners for peer
-assessment, it is also removed from their queue. However, if any learner has
-already graded the inappropriate response, it is counted as one of the
-submissions they have graded.
-
-.. note:: After you remove an inappropriate response from peer assessment, you
-   decide whether the learner who submitted that response is allowed to submit
-   a replacement response. If you do not want to allow the learner to submit a
-   replacement response, you do not need to take any additional action. The
-   learner receives a grade of zero for the entire submission. To allow the
-   learner to resubmit a response for a canceled submission, :ref:`reset the
-   learner's attempts for the problem<reset_attempts>`.
-
-You remove a submission from peer assessment by completing these steps.
-
-#. In the LMS, go to the peer assessment assignment that contains the
-   submission you want to remove.
-
-#. Scroll to the bottom of the problem, then select the black **Course Staff
-   Information** banner.
-
-#. Scroll down to the **Get Student Info** box, enter the learner's username in
-   the box, and select **Submit**.
-
-   The learner's information appears below the **Get Student Info** box.
-
-#. Select **Remove Submission from Peer Grading**.
-
-.. image:: ../../../../shared/images/ORA_RemoveSubmission.png
-   :alt: Dialog allowing comments to be entered when removing a learner
-       submission.
-
-#. Enter a comment to explain the removal. This comment appears to the learner
-   when she views her response in the open response assessment problem.
-
-#. Select **Remove submission**.
-
-   The inappropriate submission is removed from peer assessment. When you
-   access this learner's information again, instead of the response, you see a
-   note showing the date and time that the submission was removed, and the
-   comments that you entered.
-
-   Removed submissions are also removed from the list of Top Responses if they
-   were previously listed.
-
-.. image:: ../../../../shared/images//ORA_CancelledStudentResponse.png
-   :alt: The date, time and comment for removal of a learner response is shown
-       instead of the original response.
-
-.. _Locate a specific ORA submission:
+To view metrics about learners in the assignment, including the number who
+are active in each step, follow these steps.
 
-*************************************************
-Locate a Specific Submission in an ORA Assignment
-*************************************************
+#. Open the assignment in the course.
 
-If you are alerted to an inappropriate ORA submission that you want to cancel
-and :ref:`remove from peer assessment<Remove a learner response from peer
-grading>`, locate the specific submission by following these steps.
+#. Scroll to the bottom of the assignment and select **View Assignment
+   Statistics**.
 
-#. Ask the person who reported the incident to send you a sample of text from
-   the inappropriate response.
+   You see statistics for the assignment, including the total number of
+   responses and the location ID for the assignment. Learner progress is
+   available for the steps that exist in the current assignment. Only steps
+   that exist in the assignment are shown. For each step, you can see the
+   number of learners who are currently working through (but who have not
+   completed) that step.
 
-#. Contact your edX Program Manager to request a data download of ORA
-   responses for your course.
+   You can see the number of learners who are in each of the following states
+   in the assignment.
 
-   You will receive the download as a spreadsheet or in .csv file format.
+     * Waiting to assess responses or to receive grades
+     * Completed training, self, or peer assessments
+     * Have had their responses cancelled
 
-#. Search the spreadsheet for text that matches the sample text from the
-   inappropriate response.
+In addition, you can see the release and due dates for each problem step in the
+assignment.
 
-#. From any matching entries in the spreadsheet, locate the username of the
-   learner who posted the submission.
+.. image:: ../../../../shared/images/ORA_AssignmentStats.png
+   :alt: The number of active learners in each step, and the start and due
+         dates for each step are shown on the View Assignment Statistics page.
 
-#. Make a note of the username, and follow the steps to :ref:`remove a learner
-   response from peer grading<Remove a learner response from peer grading>`.