Merge pull request #307 from EducationalTestingService/306-change-sec…

…tion-order Change section order
EducationalTestingService · Oct 29, 2019 · ad35063 · ad35063
2 parents b42ddb7 + 81d611c
commit ad35063
Show file tree

Hide file tree

Showing 18 changed files with 47 additions and 47 deletions.
diff --git a/rsmtool/comparer.py b/rsmtool/comparer.py
@@ -86,7 +86,7 @@ def _modify_eval_columns_to_ensure_version_compatibilty(df,
                                                             raise_warnings=True):
         """
         This is a helper method to ensure that the column names for eval data frames
-        are forward and backward compatible. There are two major changes in RSMTool (>6.1)
+        are forward and backward compatible. There are two major changes in RSMTool (7.0 or greater)
         that necessitate this: (1) for subgroup calculations, 'DSM' is now used
         instead of 'SMD', and (2) QWK statistics are now calculated on un-rounded scores.
         Thus, we need to check these columns to see which sets of names are being used.
@@ -145,7 +145,7 @@ def _modify_eval_columns_to_ensure_version_compatibilty(df,
                                       for col in short_metrics_list_new]
 
             if raise_warnings:
-                warnings.warn("Please note that newer versions of RSMTool (>6.1) use only the trimmed "
+                warnings.warn("Please note that newer versions of RSMTool (7.0 or greater) use only the trimmed "
                               "scores for weighted kappa calculations. Comparisons with experiments using "
                               "`trim_round` for weighted kappa calculations will be deprecated in the next "
                               "major release.", category=DeprecationWarning)
@@ -479,8 +479,8 @@ def load_rsmtool_output(self, filedir, figdir, experiment_id, prefix, groups_eva
                 # if `SMD` is being used, rather than `DSM`, we print a note for the user; we don't
                 # want to go so far as to raise a warning, but we do want to give the user some info
                 if smd_name == 'SMD':
-                    warnings.warn("The the subgroup evaluations in `{}` use 'SMD'. Please note "
-                                  "that newer versions of RSMTool (>6.1) use 'DSM' with subgroup "
+                    warnings.warn("The subgroup evaluations in `{}` use 'SMD'. Please note "
+                                  "that newer versions of RSMTool (7.0 or greater) use 'DSM' with subgroup "
                                   "evaluations. For additional details on how these metrics "
                                   "differ, see the RSMTool documentation. Comparisons with experiments "
                                   "using SMD for subgroup calculations will be deprecated in the next major "

diff --git a/rsmtool/notebooks/builtin_model.ipynb b/rsmtool/notebooks/builtin_model.ipynb
@@ -79,7 +79,7 @@
     "collapsed": true
    },
    "source": [
-    "### Standardized and Relative Regression Coefficients (Betas)\n",
+    "### Standardized and relative regression coefficients (betas)\n",
     "\n",
     "The relative coefficients are intended to show relative contribution of different feature and their primary purpose is to indentify whether one of the features has an unproportionate effect over the final score. They are computed as standardized/(sum of absolute values of standardized coefficients). \n",
     "\n",
@@ -245,7 +245,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/evaluation.ipynb b/rsmtool/notebooks/comparison/evaluation.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Association Statistics"
+    "## Association statistics"
    ]
   },
   {

diff --git a/rsmtool/notebooks/comparison/feature_descriptives.ipynb b/rsmtool/notebooks/comparison/feature_descriptives.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Overall Descriptive Feature Statistics"
+    "## Overall descriptive feature statistics"
    ]
   },
   {
@@ -25,7 +25,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Prevalence of Recoded Cases"
+    "### Prevalence of recoded cases"
    ]
   },
   {
@@ -46,7 +46,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Feature Value Distribution"
+    "### Feature value distribution"
    ]
   },
   {

diff --git a/rsmtool/notebooks/comparison/features_by_group.ipynb b/rsmtool/notebooks/comparison/features_by_group.ipynb
@@ -9,7 +9,7 @@
    "outputs": [],
    "source": [
     "for group in groups_eval:\n",
-    "    display(Markdown('### Feature Value Box Plots by {}'.format(group)))\n",
+    "    display(Markdown('### Feature value box plots by {}'.format(group)))\n",
     "    markdown_str = \"All plots for the old model ({}) appears first followed by all the plots for the new model ({}).\".format(experiment_id_old, experiment_id_new)\n",
     "    display(Markdown(markdown_str))\n",
     "\n",
@@ -61,7 +61,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/model.ipynb b/rsmtool/notebooks/comparison/model.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Feature Weights"
+    "## Feature weights"
    ]
   },
   {
@@ -92,7 +92,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/pca.ipynb b/rsmtool/notebooks/comparison/pca.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Data Reduction Analyses"
+    "## Data reduction analyses"
    ]
   },
   {
@@ -15,7 +15,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Principal Component Analysis for the New Model ({})'''.format(experiment_id_old)\n",
+    "markdown_str = '''### Principal component analysis for the new model ({})'''.format(experiment_id_old)\n",
     "Markdown(markdown_str)"
    ]
   },
@@ -76,7 +76,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/preprocessed_features.ipynb b/rsmtool/notebooks/comparison/preprocessed_features.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Feature Distributions and Inter-feature Correlations"
+    "## Feature distributions and inter-feature correlations"
    ]
   },
   {
@@ -15,7 +15,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Distributions of the Standardized (if applicable), Truncated Feature Values for the Old Model ({})'''.format(experiment_id_old)"
+    "markdown_str = '''### Distributions of the standardized (if applicable), truncated feature values for the old model ({})'''.format(experiment_id_old)"
    ]
   },
   {
@@ -44,7 +44,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Distributions of the Standardized (if applicable), Truncated Feature Values for the New Model ({})'''.format(experiment_id_new)"
+    "markdown_str = '''### Distributions of the standardized (if applicable), truncated feature values for the new model ({})'''.format(experiment_id_new)"
    ]
   },
   {
@@ -73,7 +73,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Differences in Inter-feature correlations between the old model ({}) and the new model ({})'''.format(experiment_id_old, experiment_id_new)\n"
+    "markdown_str = '''### Differences in inter-feature correlations between the old model ({}) and the new model ({})'''.format(experiment_id_old, experiment_id_new)\n"
    ]
   },
   {
@@ -93,14 +93,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Human-Feature Correlations"
+    "## Human-feature correlations"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Overall Human-Feature Marginal Correlations"
+    "### Overall human-feature marginal correlations"
    ]
   },
   {
@@ -121,7 +121,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Overall Human-Feature Partial Correlations"
+    "### Overall human-feature partial correlations"
    ]
   },
   {
@@ -156,7 +156,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/preprocessed_features_by_group.ipynb b/rsmtool/notebooks/comparison/preprocessed_features_by_group.ipynb
@@ -10,7 +10,7 @@
    "source": [
     "for group in groups_eval:\n",
     "    if not out_dfs['mcor_sc1_by_{}'.format(group)].empty:\n",
-    "        markdown_str = '''### Human-Feature Marginal Correlations by {}'''.format(group)\n",
+    "        markdown_str = '''### Human-feature marginal correlations by {}'''.format(group)\n",
     "        display(Markdown(markdown_str))\n",
     "        display(HTML(out_dfs['mcor_sc1_by_{}'.format(group)].to_html(index=True, classes=['alternate_colors3_groups'], float_format=float_format_func)))"
    ]
@@ -25,7 +25,7 @@
    "source": [
     "for group in groups_eval:\n",
     "    if not out_dfs['pcor_sc1_by_{}'.format(group)].empty:\n",
-    "        markdown_str = '''### Human-Feature Partial Correlations by {}'''.format(group)\n",
+    "        markdown_str = '''### Human-feature partial correlations by {}'''.format(group)\n",
     "        display(Markdown(markdown_str))\n",
     "        display(HTML(out_dfs['pcor_sc1_by_{}'.format(group)].to_html(index=True, classes=['alternate_colors3_groups'], float_format=float_format_func)))"
    ]
@@ -48,7 +48,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/comparison/score_distributions.ipynb b/rsmtool/notebooks/comparison/score_distributions.ipynb
@@ -4,14 +4,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "##  Score Distributions"
+    "##  Score distributions"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Score Distribution (Percentage)"
+    "### Score distribution (percentage)"
    ]
   },
   {
@@ -36,7 +36,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Confusion Matrix (Human-Machine) for old model ({})'''.format(experiment_id_old)\n"
+    "markdown_str = '''### Confusion matrix (human-machine) for old model ({})'''.format(experiment_id_old)\n"
    ]
   },
   {
@@ -60,7 +60,7 @@
    },
    "outputs": [],
    "source": [
-    "markdown_str = '''### Confusion Matrix (Human-Machine) for new model ({})'''.format(experiment_id_new)\n"
+    "markdown_str = '''### Confusion matrix (human-machine) for new model ({})'''.format(experiment_id_new)\n"
    ]
   },
   {
@@ -85,7 +85,7 @@
    "outputs": [],
    "source": [
     "if not (outputs_new[\"df_confmatrix\"].empty or outputs_old[\"df_confmatrix\"].empty):\n",
-    "    display(Markdown('### Confusion Matrix (Human-Machine) (change)'))\n",
+    "    display(Markdown('### Confusion matrix (human-machine) (change)'))\n",
     "    df_change = outputs_new[\"df_confmatrix\"] - outputs_old[\"df_confmatrix\"]\n",
     "    display(HTML(df_change.to_html(index=True, classes=['alternate_colors2'])))"
    ]
@@ -108,7 +108,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.4"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/dff_by_group.ipynb b/rsmtool/notebooks/dff_by_group.ipynb
@@ -7,7 +7,7 @@
    "outputs": [],
    "source": [
     "if len(groups_desc) > 0:\n",
-    "    markdown_str = [\"## Differential Feature functioning\"]\n",
+    "    markdown_str = [\"## Differential feature functioning\"]\n",
     "    markdown_str.append(\"This section shows differential feature functioning (DFF) plots \"\n",
     "                        \"for all features and subgroups. The features are shown after applying \"\n",
     "                        \"transformations (if applicable) and truncation of outliers.\")\n",

diff --git a/rsmtool/notebooks/evaluation.ipynb b/rsmtool/notebooks/evaluation.ipynb
@@ -115,7 +115,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Confusion Matrix"
+    "### Confusion matrix"
    ]
   },
   {

diff --git a/rsmtool/notebooks/fairness_analyses.ipynb b/rsmtool/notebooks/fairness_analyses.ipynb
@@ -82,7 +82,7 @@
     "\n",
     "for group in groups_eval:\n",
     "    \n",
-    "    display(Markdown(\"### Fairness evaluation by {}\".format(group)))\n",
+    "    display(Markdown(\"### Additional fairness evaluation by {}\".format(group)))\n",
     "    \n",
     "    # run the actual analyses. We are currently doing this in the notebook\n",
     "    # so that it is easy to skip these if necessary. The notebook\n",
@@ -243,7 +243,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.9"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/intermediate_file_paths.ipynb b/rsmtool/notebooks/intermediate_file_paths.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Links to Intermediate Files"
+    "## Links to intermediate files"
    ]
   },
   {
@@ -49,7 +49,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/preprocessed_features.ipynb b/rsmtool/notebooks/preprocessed_features.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "##  Feature Distributions and Inter-feature Correlations"
+    "##  Feature distributions and inter-feature correlations"
    ]
   },
   {
@@ -292,7 +292,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/summary/intermediate_file_paths.ipynb b/rsmtool/notebooks/summary/intermediate_file_paths.ipynb
@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Links to Intermediate Files"
+    "## Links to intermediate files"
    ]
   },
   {
@@ -51,7 +51,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.3"
+   "version": "3.6.7"
   }
  },
  "nbformat": 4,

diff --git a/rsmtool/notebooks/summary/preprocessed_features.ipynb b/rsmtool/notebooks/summary/preprocessed_features.ipynb
@@ -41,7 +41,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Marginal and Partial correlations\n",
+    "## Marginal and partial correlations\n",
     "\n",
     "The tables below shows correlations between truncated and standardized (if applicable) values of each feature against human score for each model. All correlations are computed on the training sets."
    ]