Skip to content

Commit

Permalink
Merge pull request #405 from edx/carol/ORA_doc_enh
Browse files Browse the repository at this point in the history
ORA doc enhancements incl best practices
  • Loading branch information
catong committed Jul 14, 2015
2 parents afc9056 + 60c9281 commit 1dc4f02
Show file tree
Hide file tree
Showing 5 changed files with 441 additions and 260 deletions.
12 changes: 6 additions & 6 deletions en_us/course_authors/source/front_matter/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ July 2015

* - Date
- Change
* - 15 July 2015
- Added the :ref:`Best Practices for ORA` and :ref:`PA Scoring` sections.
* - 8 July 2015
- Added the :ref:`Poll Tool` and :ref:`Survey Tool` sections.
* - 1 July
Expand Down Expand Up @@ -202,13 +204,11 @@ February 2015
- Updated the :ref:`Cohorted Courseware Overview` section to reflect the
ability to delete content groups and view their usage in a course.
* - 2/19/15
- Updated ORA documentation to reflect ability to :ref:`remove a student
response from peer grading<Remove a student response from peer
grading>`.
- Updated ORA documentation to reflect ability to :ref:`Remove a learner response from peer grading`
* -
- Updated ORA documentation to indicate that course staff can use student
usernames to :ref:`access student information<Access Student
Information>`. They no longer need to obtain an anonymized student ID.
- Updated ORA documentation to indicate that course staff can use learner
usernames to :ref:`access learner information<Access Learner
Information>`. They no longer need to obtain an anonymized learner ID.
* - 2/13/15
- Updated the example in the :ref:`Drag and Drop Problem XML` topic.
* - 2/12/15
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -259,11 +259,16 @@ Set the Assignment Type and Due Date for a Subsection
You set the assignment type for problems at the subsection level.

When you set the assignment type for a subsection, all problems within the
subsection are graded and weighted as a single type. For example, if you
subsection are graded and weighted as a single type. For example, if you
designate the assignment type for a subsection as **Homework**, then all
problem types in that subsection are graded as homework.

To set the assignment type and due date for a subsection:
.. note:: Unlike other problem types, ORA assignments are not governed by the
subsection due date. Due dates for each ORA assignment are set in the assignment's
settings. For details, see :ref:`PA Specify Name and Dates`.


To set the assignment type and due date for a subsection, follow these steps.

#. Click the Settings icon in the subsection box.

Expand Down
118 changes: 59 additions & 59 deletions en_us/shared/subsections/open_response_assessments/Access_ORA_Info.rst
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
.. _Accessing ORA Assignment Information:

##########################################
Accessing Assignment and Student Metrics
Accessing Assignment and Learner Metrics
##########################################

After your open response assessment assignment has been released, you can access
information about the number of students in each step of the assignment or the
performance of individual students. This information is available in the
information about the number of learners in each step of the assignment or the
performance of individual learners. This information is available in the
**Course Staff Information** section at the end of each assignment. To access
student information, open the assignment in the courseware, scroll to the bottom of the
assignment, and then click the black **Course Staff Information** banner.
learner information, open the assignment in the courseware, scroll to the bottom of the
assignment, and then select the black **Course Staff Information** banner.

.. image:: ../../../../shared/building_and_running_chapters/Images/PA_CourseStaffInfo_Collapsed.png
:alt: The Course Staff Information banner at the bottom of the peer assessment

When you access a specific student's information for an open response
When you access a specific learner's information for an open response
assessment, you can view his responses and, if necessary, :ref:`cancel the
student's submission<Remove a student response from peer grading>` so that it is
learner's submission<Remove a learner response from peer grading>` so that it is
not included in peer assessments.

.. _PA View Metrics for Individual Steps:
Expand All @@ -25,7 +25,7 @@ not included in peer assessments.
View Metrics for Individual Steps
************************************************

You can check the number of students who have completed, or are currently
You can check the number of learners who have completed, or are currently
working through, the following steps:

* Submitted responses.
Expand All @@ -35,150 +35,150 @@ working through, the following steps:
* Completed the entire assignment.

To find this information, open the assignment in the courseware, scroll to the
bottom of the assignment, and then click **Course Staff Information**.
bottom of the assignment, and then select **Course Staff Information**.

The **Course Staff Information** section expands, and you can see the number
of students who are currently working through (but have not completed) each
of learners who are currently working through (but have not completed) each
step of the problem.

.. image:: ../../../../shared/building_and_running_chapters/Images/PA_CourseStaffInfo_Expanded.png
:alt: The Course Staff Information box expanded, showing problem status

.. _Access Information for a Specific Student:
.. _Access Information for a Specific Learner:

***********************************************
Access Information for a Specific Student
Access Information for a Specific Learner
***********************************************

You can access information about an individual student's performance on a peer
You can access information about an individual learner's performance on a peer
assessment assignment, including:

* The student's response.
* The peer assessments that other students performed on the student's
* The learner's response.
* The peer assessments that other learners performed on the learner's
response, including feedback on individual criteria and on the overall
response.
* The peer assessments that the student performed on other students'
* The peer assessments that the learner performed on other learners'
responses, including feedback on individual criteria and on the overall
responses.
* The student's self assessment.
* The learner's self assessment.

In the following example, you can see the performance information for a specific
student. This student's response received one peer assessment, and the student
completed a peer assessment on one other student's response. The student also
learner. This learner's response received one peer assessment, and the learner
completed a peer assessment on one other learner's response. The learner also
completed a self assessment.

.. image:: ../../../../shared/building_and_running_chapters/Images/PA_SpecificStudent.png
:width: 500
:alt: Report showing information about a student's response
:alt: Report showing information about a learner's response

To determine whether this student has received the required number of
assessments from other students and has completed the required number of
assessments for other students, refer to the **Graded By** and **Must Grade**
To determine whether this learner has received the required number of
assessments from other learners and has completed the required number of
assessments for other learners, refer to the **Graded By** and **Must Grade**
values that were set for the open response assessment assignment in Studio. For
more information about these settings, see :ref:`Specify Step Settings<PA
Specify Step Settings>`.

For details about accessing information for a specific student, and for an
example that shows a student's response with more assessments, see :ref:`Access
Student Information`.
For details about accessing information for a specific learner, and for an
example that shows a learner's response with more assessments, see :ref:`Access
Learner Information`.


.. _Access Student Information:
.. _Access Learner Information:

=======================================
Access a Specific Student's Information
Access a Specific Learner's Information
=======================================

#. In the LMS, go to the peer assessment assignment that you want to see.

#. Scroll to the bottom of the problem, and click the black **Course Staff
#. Scroll to the bottom of the problem, and select the black **Course Staff
Information** banner.

#. In the **Get Student Info** box, enter the student's username, and click
#. In the **Get Student Info** box, enter the learner's username, and select
**Submit**.

The student's information appears below the **Get Student Info** box.
The learner's information appears below the **Get Student Info** box.

The following example shows:

* The student's response.
* The learner's response.
* The two peer assessments for the response.
* The two peer assessments the student completed.
* The student's self assessment.
* The two peer assessments the learner completed.
* The learner's self assessment.

For a larger view, click the image so that it opens by itself in the browser
For a larger view, select the image so that it opens by itself in the browser
window, and then click anywhere on the image that opens.

.. image:: ../../../../shared/building_and_running_chapters/Images/PA_SpecificStudent_long.png
:width: 250
:alt: Report showing information about a student's response
:alt: Report showing information about a learner's response


.. _Remove a student response from peer grading:
.. _Remove a learner response from peer grading:

************************************************
Remove a Student Response from Peer Grading
Remove a Learner's Response from Peer Grading
************************************************

If you use open response assessments, students might alert you to vulgar,
If you use open response assessments, learners might alert you to vulgar,
abusive, or otherwise inappropriate responses that they have seen while
performing peer assessments. In such a situation you can :ref:`locate<Locate a
specific ORA submission>` and cancel the submission. Doing so removes the
inappropriate response from peer assessments so that it is no longer shown to
other students.
other learners.

.. note:: Removing a student's submission is an irreversible action.
.. note:: Removing a learner's submission is an irreversible action.

When you cancel an inappropriate submission, the response is immediately removed
from the pool of submissions available for peer assessment. If the inappropriate
response has already been sent to other students for peer assessment, it is also
removed from their queue. However, if any student has already graded the
response has already been sent to other learners for peer assessment, it is also
removed from their queue. However, if any learner has already graded the
inappropriate response, it is counted as one of the submissions they have
graded.

.. note:: After you remove an inappropriate response from peer assessment, you
decide whether the student who submitted that response is allowed to submit a
replacement response. If you do not want to allow the student to submit a
decide whether the learner who submitted that response is allowed to submit a
replacement response. If you do not want to allow the learner to submit a
replacement response, you do not need to take any additional action. The
student receives a grade of zero for the entire submission. To allow the
student to resubmit a response for a cancelled submission, :ref:`reset the
student's attempts for the problem<reset_attempts>`.
learner receives a grade of zero for the entire submission. To allow the
learner to resubmit a response for a cancelled submission, :ref:`reset the
learner's attempts for the problem<reset_attempts>`.

Remove a submission from peer assessment by completing these steps.

#. In the LMS, go to the peer assessment assignment that contains the submission
you want to remove.

#. Scroll to the bottom of the problem, then click the black **Course Staff
#. Scroll to the bottom of the problem, then select the black **Course Staff
Information** banner.

#. Scroll down to the **Get Student Info** box, enter the student's username in
the box, and click **Submit**.
#. Scroll down to the **Get Student Info** box, enter the learner's username in
the box, and select **Submit**.

The student's information appears below the **Get Student Info** box.
The learner's information appears below the **Get Student Info** box.

#. Scroll down to the **Student Response** section and locate the submission you
want to remove.

.. image:: ../../../../shared/building_and_running_chapters/Images/ORA_RemoveSubmission.png
:alt: Dialog allowing comments to be entered when removing a student submission
:alt: Dialog allowing comments to be entered when removing a learner submission

5. Enter a comment to document or explain the removal. This comment appears to
the student when she views her response in the open response assessment
the learner when she views her response in the open response assessment
problem.

#. Click **Remove submission**.

The inappropriate submission is removed from peer assessment. When you access
this student's information again, instead of the response, you see a note
this learner's information again, instead of the response, you see a note
showing the date and time that the submission was removed, and the comments
that you entered.

Removed submissions are also removed from the list of Top Responses if they
were previously listed.

.. image:: ../../../../shared/building_and_running_chapters/Images//ORA_CancelledStudentResponse.png
:alt: The date, time and comment for removal of a student response is shown instead of the original response.
:alt: The date, time and comment for removal of a learner response is shown instead of the original response.


.. _Locate a specific ORA submission:
Expand All @@ -188,7 +188,7 @@ Locate a Specific Submission in an ORA Assignment
*************************************************

If you are alerted to an inappropriate ORA submission that you want to cancel
and :ref:`remove from peer assessment<Remove a student response from peer
and :ref:`remove from peer assessment<Remove a learner response from peer
grading>`, locate the specific submission by following these steps.

#. Ask the person who reported the incident to send you a sample of text from
Expand All @@ -205,8 +205,8 @@ grading>`, locate the specific submission by following these steps.
#. From any matching entries in the spreadsheet, locate the username of the
learner who posted the submission.

#. Make a note of the username, and follow the steps to :ref:`remove a student
response from peer grading<Remove a student response from peer grading>`.
#. Make a note of the username, and follow the steps to :ref:`remove a learner
response from peer grading<Remove a learner response from peer grading>`.



0 comments on commit 1dc4f02

Please sign in to comment.