Merge 59351b6 into 4b15297

README.rst

@@ -41,11 +41,10 @@ Rater Scoring Modeling Tool (RSMTool) is a python package which automates and co
 
 We expect the primary users of RSMTool to be researchers working on developing new automated scoring engines or on improving existing ones. Note that RSMTool is not a scoring engine by itself but rather a tool for building and evaluating machine learning models that may be used in such engines. 
 
-RSMTool is driven by a configuration file that users have to supply. Given the large number of available options, this can get complicated especially for new users. That's why RSMTool can help users generate configuration files interactively via guided prompts! Click the image below to see a demo video of this feature.
+RSMTool is driven by a configuration file that users have to supply. Given the large number of available options, this can get complicated especially for new users. That's why RSMTool can help users generate configuration files interactively via guided prompts! The video below demonstrates this feature.
 
-.. raw:: html
-
-   <a href="https://asciinema.org/a/313107" target="_blank"><img align="center" src="https://asciinema.org/a/313107.svg" width="500"/></a>
+.. image:: https://raw.githubusercontent.com/EducationalTestingService/rsmtool/master/doc/assets/demo.gif
+   :alt: License
 
 Getting Started
 ---------------

conda-recipe/rsmtool/meta.yaml

@@ -65,5 +65,5 @@ test:
 
 about:
   home: https://github.com/EducationalTestingService/rsmtool
-  license: Apache 2.0
+  license: Apache-2.0
   license_file: LICENSE

doc/add_dash_anchors.py

@@ -23,7 +23,8 @@
 from bs4 import BeautifulSoup
 
 # pre-define the list of HTML files we want to miodify
-FILES_TO_MODIFY = ["advanced_usage.html", "api.html", "contributing.html",
+FILES_TO_MODIFY = ["advanced_usage.html", "api.html", "builtin_models.html",
+                   "automated_configuration.html", "contributing.html",
                    "custom_notebooks.html", "evaluation.html",
                    "getting_started.html", "pipeline.html",
                    "internal.html", "tutorial.html",

doc/evaluation.rst

@@ -89,10 +89,10 @@ where:
 
         * :math:`n_{+k}` - total number of responses where :math:`M_i=k` 
 
-Kappa is computed using `skll.metrics.kappa <https://skll.readthedocs.io/en/latest/api/skll.html#from-metrics-module>`_ with ``weights`` set to ``None`` and ``allow_off_by_one`` set to ``False`` (default).
+Kappa is computed using `skll.metrics.kappa <https://skll.readthedocs.io/en/latest/api/metrics.html#skll.metrics.kappa>`_ with ``weights`` set to ``None`` and ``allow_off_by_one`` set to ``False`` (default).
 
 .. note::
-   See `this discussion <https://github.com/EducationalTestingService/skll/issues/391#issuecomment-444145567>`_ for the explanation of how the `SKLL implementation <https://skll.readthedocs.io/en/latest/api/skll.html#skll.kappa>`_  differs from the `scikit-learn implementation <https://scikit-learn.org/stable/modules/generated/sklearn.metrics.cohen_kappa_score.html>`_. The two implementations might produce different results if the matrix contains missing labels. For example, consider the hypothetical scenario where our predictions only contain the labels ``1``, ``2``, and ``4``. In the SKLL implementation, the missing ``3``  will be automatically added to the list of labels whereas in the scikit-learn implementation, the ``3`` would only be added if a complete list of labels was passed to the function via the optional ``labels`` keyword argument.
+   See `this discussion <https://github.com/EducationalTestingService/skll/issues/391#issuecomment-444145567>`_ for the explanation of how the `SKLL implementation <https://skll.readthedocs.io/en/latest/api/metrics.html#skll.metrics.kappa>`_  differs from the `scikit-learn implementation <https://scikit-learn.org/stable/modules/generated/sklearn.metrics.cohen_kappa_score.html>`_. The two implementations might produce different results if the matrix contains missing labels. For example, consider the hypothetical scenario where our predictions only contain the labels ``1``, ``2``, and ``4``. In the SKLL implementation, the missing ``3``  will be automatically added to the list of labels whereas in the scikit-learn implementation, the ``3`` would only be added if a complete list of labels was passed to the function via the optional ``labels`` keyword argument.
 
 
 .. _qwk:
@@ -114,7 +114,7 @@ See `Haberman (2019) <https://onlinelibrary.wiley.com/doi/abs/10.1002/ets2.12258
 
 .. note::
 
-	In RSMTool v6.x and earlier QWK was computed using `skll.metrics.kappa <https://skll.readthedocs.io/en/latest/api/skll.html#from-metrics-module>`_ with ``weights`` set to ``"quadratic"``. Continuous scores were rounded for computation. Both formulas produce the same scores for discrete (rounded scores) but QWK values for continuous scores computed by RSMTool starting with v7.0 will be *different* from those computed by earlier versions.
+	In RSMTool v6.x and earlier QWK was computed using `skll.metrics.kappa <https://skll.readthedocs.io/en/latest/api/metrics.html#skll.metrics.kappa>`_ with ``weights`` set to ``"quadratic"``. Continuous scores were rounded for computation. Both formulas produce the same scores for discrete (rounded scores) but QWK values for continuous scores computed by RSMTool starting with v7.0 will be *different* from those computed by earlier versions.
 
 
 .. _r: 

doc/getting_started.rst

@@ -2,7 +2,7 @@
 
 Installation
 ============
-Note that RSMTool currently only works with Python 3.6.x.
+Note that RSMTool currently works with Python 3.6, 3.7, and 3.8
 
 Installing with conda
 ----------------------

doc/internal/release_process.rst

@@ -19,19 +19,19 @@ This process is only meant for the project administrators, not users and develop
 
    e. update the README and this release documentation, if necessary.
 
-4. Build the PyPI source distribution using ``python setup.py sdist build``.
+4. Build the PyPI source and wheel distributions using ``python setup.py sdist build`` and ``python setup.py bdist_wheel build`` respectively.
 
-5. Upload the source distribution to TestPyPI  using ``twine upload --repository testpypi dist/*``. You will need to have the ``twine`` package installed and set up your ``$HOME/.pypirc`` correctly. See details `here <https://packaging.python.org/guides/using-testpypi/>`__.
+5. Upload the source and wheel distributions to TestPyPI  using ``twine upload --repository testpypi dist/*``. You will need to have the ``twine`` package installed and set up your ``$HOME/.pypirc`` correctly. See details `here <https://packaging.python.org/guides/using-testpypi/>`__.
 
 6. Install the TestPyPI package as follows::
 
     pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple rsmtool
 
 7. Then run some tests from a RSMTool working copy. If the TestPyPI package works, then move on to the next step. If it doesn't, figure out why and rebuild and re-upload the package.
 
-8. Build the new generic conda package by running the following command in the ``conda-recipe`` directory (note that this assumes that you have cloned RSMTool in a directory named ``rsmtool``)::
+8. Build the new generic conda package by running the following command in the ``conda-recipe`` directory (note that this assumes that you have cloned RSMTool in a directory named ``rsmtool`` and that the latest version of ``numpy`` is ``1.18``)::
 
-    conda build -c conda-forge -c ets .
+    conda build -c conda-forge -c ets --numpy=1.18 .
 
 9. Upload the package to anaconda.org using ``anaconda upload --user ets <package tarball>``. You will need to have the appropriate permissions for the ``ets`` organization. 
 

rsmtool/version.py

@@ -3,5 +3,5 @@
 in one place. Based on the suggestion `here. <http://bit.ly/16LbuJF>`_
 """
 
-__version__ = '7.1.0'
+__version__ = '8.0.0'
 VERSION = tuple(int(x) for x in __version__.split('.'))

tests/test_preprocessor.py

@@ -1,5 +1,3 @@
-import warnings
-
 import numpy as np
 import pandas as pd