Merge branch 'ucbds-infra:master' into truncate-traceback

CHANGELOG.md

@@ -1,8 +1,40 @@
 # Changelog
 
-**Unreleased:**
+**v4.3.1:**
+
+* Change `sklearn` to `scikit-learn` in requirements files generated by Otter Generate
+
+**4.3.0:**
+
+* Support dictionary and tuple assignments in solution substitution in Otter Assign per [#587](https://github.com/ucbds-infra/otter-grader/issues/587)
+* Add the `pdf` argument to `ottr::export` in R assignments created with Otter Assign per [#440](https://github.com/ucbds-infra/otter-grader/issues/440)
+
+**v4.2.1:**
+
+* Update Otter Grade image tag when the base image changes per [#583](https://github.com/ucbds-infra/otter-grader/issues/583)
+* Make gspread an optional dependency per [#577](https://github.com/ucbds-infra/otter-grader/issues/577)
+
+**v4.2.0:**
+
+* Added configuration files for Otter Assign per [#565](https://github.com/ucbds-infra/otter-grader/issues/565)
+* Fixed log execution bug from Slack as described in [#571](https://github.com/ucbds-infra/otter-grader/pull/571)
+
+**v4.1.2:**
+
+* Added `ipython` to `install_requires`
+
+**v4.1.1:**
+
+* Added `MANIFEST.in`
+
+**v4.1.0:**
 
 * Display emojis with test results to more clearly show which tests pass and which fail, as per [#533](https://github.com/ucbds-infra/otter-grader/pull/533).
+* Allow users to specify a Python version in Otter Generate and Otter Assign
+* Round `results.json` point values to 5 decimal places per [#538](https://github.com/ucbds-infra/otter-grader/issues/538)
+* Optionally display PDF generation/submission failures to students via `results.json` per [#494](https://github.com/ucbds-infra/otter-grader/issues/494)
+* Added the `force_public_test_summary` key to the autograder config per [#539](https://github.com/ucbds-infra/otter-grader/issues/539)
+* Made the "Public Tests" section on Gradescope appear as failing when not all public tests passed per [#539](https://github.com/ucbds-infra/otter-grader/issues/539)
 
 **v4.0.2:**
 

CITATION.cff

@@ -8,10 +8,10 @@ authors:
   -
     name: UC Berkeley Data Science Education Program
 cff-version: "1.1.0"
-date-released: 2022-09-20
+date-released: 2023-02-14
 license: "BSD-3-Clause"
 message: "If you use this software, please cite it as below."
 repository-code: "https://github.com/ucbds-infra/otter-grader"
 title: "Otter-Grader: A Python and R autograding solution"
-version: "4.0.2"
+version: "4.3.1"
 doi: 10.5281/zenodo.5259955

Dockerfile

@@ -52,5 +52,5 @@ RUN mkdir /autograder
 ADD requirements.txt /tmp/requirements.txt
 RUN pip install -r /tmp/requirements.txt
 
-RUN pip install otter-grader==4.0.2
+RUN pip install otter-grader==4.3.1
 RUN Rscript -e "install.packages('ottr', dependencies=TRUE, repos='http://cran.us.r-project.org')"

MANIFEST.in

@@ -0,0 +1,18 @@
+include requirements.txt
+exclude .coveragerc
+exclude .dockerignore
+exclude .gitattributes
+exclude .github
+exclude .gitignore
+exclude .readthedocs.yml
+exclude codecov.yml
+exclude Dockerfile
+exclude docs
+exclude environment.yml
+exclude examples
+exclude logo
+exclude Makefile
+exclude release.py
+exclude rtd_environment.yml
+exclude slack-shields.json
+exclude test

codecov.yml

@@ -6,5 +6,3 @@ coverage:
     project:
       default:
         threshold: 100%
-comment:
-  behavior: new

docs/otter_assign/v1/notebook_format.rst

@@ -45,6 +45,18 @@ defaults) is ``export_cell``.
 
 .. fica:: otter.assign.assignment.Assignment
 
+For assignments that share several common configurations, these can be specified in a separate YAML
+file whose path is passed to the ``config_file`` key. When this key is encountered, Otter will read
+the file and load the configurations defined therein into the assignment config for the notebook
+it's running. Any keys specified in the notebook itself will override values in the file.
+
+.. code-block:: yaml
+
+    # ASSIGNMENT CONFIG
+    config_file: ../assignment_config.yml
+    files:
+        - data.csv
+
 All paths specified in the configuration should be **relative to the directory containing the master 
 notebook**. If, for example, you were running Otter Assign on the ``lab00.ipynb`` notebook in the 
 structure below:
@@ -98,25 +110,27 @@ their values:
         show_hidden: true
 
 You can also set the autograder up to automatically upload PDFs to student submissions to another 
-Gradescope assignment by setting the necessary keys in the ``pdfs`` subkey of ``generate``:
+Gradescope assignment by setting the necessary keys under ``generate``:
 
 .. code-block:: yaml
 
     generate:
-        token: ''
+        token: YOUR_TOKEN      # optional
         course_id: 1234        # required
         assignment_id: 5678    # required
         filtering: true        # true is the default
 
-If you don't specify a token, you will be prompted for your username and password when you run Otter
-Assign; optionally, you can specify these via the command line with the ``--username`` and 
-``--password`` flags. You can also run the following to retrieve your token:
+You can run the following to retrieve your token:
 
 .. code-block:: python
 
     from otter.generate.token import APIClient
     print(APIClient.get_token())
 
+If you don't specify a token, you will be prompted for your username and password when you run Otter
+Assign; optionally, you can specify these via the command line with the ``--username`` and
+``--password`` flags.
+
 Any configurations in your ``generate`` key will be put into an ``otter_config.json`` and used when
 running Otter Generate.
 
@@ -163,6 +177,14 @@ name in the metadata.
 You can find more information about how Otter performs assignment name verification
 :ref:`here<workflow_execution_submissions_assignment_name_verification>`.
 
+By default, Otter's grading images uses Python 3.7. If you need a different version, you can
+specify one using the ``python_version`` config:
+
+.. code-block:: yaml
+
+    # ASSIGNMENT CONFIG
+    python_version: 3.9
+
 
 .. _otter_assign_v1_seed_variables:
 

docs/workflow/executing_submissions/gradescope.rst

@@ -32,7 +32,6 @@ the information about test output visibility:
 
 
 .. image:: images/gradescope_autograder_output.png
-    :target: images/gradescope_autograder_output.png
     :alt: 
 
 If ``show_stdout`` was true in your ``otter_config.json``, then the autograder 
@@ -43,18 +42,16 @@ that they failed some hidden test in the printed DataFrame from the stdout.
 Below the autograder output, each test case is broken down into boxes. The first box is called
 "Public Tests" and summarizes the results of the *public* test cases only. This box is visible to 
 students. If a student fails a hidden test but no public tests, then this box will show "All tests
-passed" for that question, even though the student failed a public test.
+passed" for that question, even though the student failed a hidden test.
 
 .. image:: images/gradescope_public_passed.png
-    :target: images/gradescope_public_passed.png
     :alt: 
 
 If the student fails a public test, then the output of the failed public test will be displayed here,
 but no information about hidden tests will be included. In the example below, the student failed a
 hidden test case in ``q1`` and a public test case in ``q4``.
 
 .. image:: images/gradescope_public_failed.png
-    :target: images/gradescope_public_failed.png
     :alt: 
 
 Below the "Public Tests" box will be boxes for each question. These are hidden from the student
@@ -65,13 +62,11 @@ become visible to students after grades are published. These boxes show the resu
 If all tests cases are passed, these boxes will indicate so:
 
 .. image:: images/gradescope_questions_passed.png
-    :target: images/gradescope_questions_passed.png
     :alt: 
 
 If the studen fails any test cases, that information will be presented here. In the example below,
 ``q1 - 4`` and ``q1 - 5`` were hidden test cases; this screenshot corresponds to the second "Public
 Tests" section screenshot above.
 
 .. image:: images/gradescope_question_failed.png
-    :target: images/gradescope_question_failed.png
     :alt: 

docs/workflow/otter_generate/container_image.rst

@@ -54,7 +54,8 @@ manually.
 ``environment.yml``
 -------------------
 
-This file specifies the conda environment that Otter creates in ``setup.sh``.
+This file specifies the conda environment that Otter creates in ``setup.sh``. By default, it uses
+Python 3.7, but this can be changed using the ``--python-version`` flag to Otter Generate.
 
 .. literalinclude:: ../../../otter/generate/templates/r/environment.yml
 

docs/workflow/otter_generate/index.rst

@@ -149,12 +149,13 @@ Gradescope container, because Otter's shelving mechanism does not store module o
 Autosubmission of Notebook PDFs
 +++++++++++++++++++++++++++++++
 
-Otter Generate allows instructors to automatically generate PDFs of students' notebooks and upload 
-these as submissions to a separate Gradescope assignment. This requires a Gradescope token, for 
-which you will be prompted to enter your Gradescope account credentials. Otter Generate also needs 
-the course ID and assignment ID of the assignment to which PDFs should be submitted—a separate 
-assignment from your autograder assignment of type "Homework / Problem Set." This 
-information can be gathered from the assignment URL on Gradescope:
+Otter Generate allows instructors to automatically generate PDFs of students' notebooks and upload
+these as submissions to a separate Gradescope assignment. This requires a Gradescope token or
+entering your Gradescope account credentials when prompted (for details see
+:ref: `the notebook metadata section <otter_assign_v1_assignment_metadata>`).
+Otter Generate also needs the course ID and assignment ID of the assignment to which PDFs should
+be submitted—a separate assignment from your autograder assignment of type "Homework / Problem Set."
+This information can be gathered from the assignment URL on Gradescope:
 
 .. code-block::
 
@@ -172,6 +173,14 @@ and password. Alternatively, you can provide these via the command-line with the
 Currently, this action supports :ref:`HTML comment filtering <pdfs>` with pagebreaks, but these 
 can be disabled with the ``filtering`` and ``pagebreaks`` keys of your config.
 
+For cases in which the generation or submission of this PDF fails, you can optionally relay this
+information to students using the ``warn_missing_pdf`` configuration. If this is set to true, a
+0-point failing test case will be displayed to the student with the error thrown while trying to
+generate or submit the PDF:
+
+.. image:: images/pdf_generation_failed_gs.png
+    :alt: PDF Generation Failed error on Gradescope
+
 
 Pass/Fail Thresholds
 ++++++++++++++++++++

examples/data-88e-proj03-exception/README.md

@@ -0,0 +1,4 @@
+# Data 88E Project 3
+
+This is an abridged version of the [Data 88E Project 3 example](../data-88e-proj03) to demonstrate
+how to write exception-based tests in the Otter Assign format.