Tutorials cleanup

.github/workflows/builddocs.yml

@@ -28,8 +28,8 @@ jobs:
         run: sudo apt-get update -y && sudo apt-get install -y pandoc
       - uses: actions/setup-python@v2
         with: 
-          python-version: '3.11' 
-      - name: Setup Python 3.11
+          python-version: '3.12' 
+      - name: Setup Python 3.12
         run: |
           python -m pip install --upgrade pip
           python -m pip install -r requirements.txt

.github/workflows/lint-and-test.yml

@@ -8,7 +8,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: ['3.8', '3.9', '3.10', '3.11']
+        python-version: ['3.9', '3.10', '3.11', '3.12']
       fail-fast: false
     steps:
     - uses: actions/checkout@v2
@@ -29,7 +29,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ['3.8', '3.9', '3.10', '3.11']
+        python-version: ['3.9', '3.10', '3.11', '3.12']
     steps:
     - uses: actions/checkout@v2
     - name: Set up Python ${{ matrix.python-version }}

docs/index.rst

@@ -25,6 +25,7 @@ production data, and current-voltage (IV) curves.
      - O&M records
      - - fill data gaps in dates and categorical records
        - visualize word clusters and patterns over time
+       - classify records using both keyword and machine learning methods
    * - timeseries
      - Production data
      - - estimate expected energy with multiple models

docs/pages/moduleguides/text.rst → docs/pages/moduleguides/guide_text.rst

@@ -18,8 +18,8 @@ to prepare data for natural language processing (text.nlp_utils submodule),
 and a visualization suite (text.visualize submodule).
 
 An example implementation of all capabilities can be found in 
-`text_class_example.py <https://github.com/sandialabs/pvOps/blob/master/tutorials/text_class_example.py>`_ 
-for specifics, and `tutorial_textmodule.ipynb <https://github.com/sandialabs/pvOps/blob/master/tutorials/tutorial_textmodule.ipynb>`_ for basics.
+`tutorial_text_class_example.py <https://github.com/sandialabs/pvOps/blob/master/tutorials/tutorial_text_class_example.py>`_ 
+for specifics, and `tutorial_text.ipynb <https://github.com/sandialabs/pvOps/blob/master/tutorials/tutorial_text.ipynb>`_ for basics.
 
 Text pre-processing
 ^^^^^^^^^^^^^^^^^^^^^
@@ -32,13 +32,13 @@ Additionally, there are options to extract dates from the text.
 * :py:func:`~pvops.text.preprocess.preprocessor` acts as a wrapper function, 
   utilizing the other preprocessing functions, which prepares the data for machine learning. 
 
-  * See ``text_class_example.prep_data_for_ML`` for an example.
+  * See ``tutorials/tutorial_text_class_example.prep_data_for_ML`` function for an example.
 
 * :py:func:`~pvops.text.preprocess.preprocessor` should be used with the keyword argument
   `extract_dates_only = True` if the primary interest is date extraction
-  instead continuing to use the data for machine learning.
+  instead of continuing to use the data for machine learning.
 
-  * See ``text_class_example.extract_dates`` module for an example.
+  * See ``tutorials/tutorial_text_class_example.extract_dates`` function for an example.
 
 
 Text classification
@@ -56,13 +56,14 @@ These functions process the O&M data to make an inference on the specified event
     :py:func:`~pvops.text.defaults.unsupervised_classifier_defs`
     functions return default values for conducting the grid search.
 
-  * See ``text_class_example.classify_supervised`` or ``text_class_example.classify_unsupervised`` 
-    modules for an example.
+  * See ``tutorials/tutorial_text_class_example.classify_supervised``
+    or ``tutorials/tutorial_text_class_example.classify_unsupervised`` 
+    functions for an example.
 
 * Once the model is built and selected, classification (for supervised ML) 
   or clustering (for unsupervised ML) analysis can be conducted on the best model returned from the pipeline object.
 
-  * See ``text_class_example.predict_best_model`` module for an example.
+  * See ``tutorials/tutorial_text_class_example.predict_best_model`` function for an example.
 
 
 Utils

docs/pages/releasenotes.rst

@@ -5,6 +5,8 @@ What's New
 
 These are new features and improvements of note in each release.
 
+.. include:: releasenotes/0.7.0.rst
+
 .. include:: releasenotes/0.6.1.rst
 
 .. include:: releasenotes/0.6.0.rst

docs/pages/releasenotes/0.7.0.rst

@@ -0,0 +1,39 @@
+0.7.0 (September 9 2025)
+------------------------
+
+This release makes updates to the text module tutorials and docstrings, as well as some miscellaneous non-breaking changes
+to associated text module functions.
+
+Functionality
+~~~~~~~~~~~~~~
+
+* All functions in the submodule `pvops.text.visualize` now accept an optional `ax` keyword argument, allowing the user to pass a matplotlib Axes instance.
+
+* All functions in the submodule `pvops.text.visualize` now return a matplotlib Figure instance (previously, some incorrectly returned Axes instances).
+
+Tutorials
+~~~~~~~~~~~~~~
+
+* Some tutorials have been renamed to better clarify their contents. All tutorial names now follow the format: `tutorial_{MODULE}_{FUNCTIONALITY}.ipynb`.
+
+* Previously, text module tutorials called functions from an additional Python script. All of these functions have been pulled into the tutorials proper.
+
+* Text tutorials have been numbered in a logical progression.
+
+* Text tutorials have had their contents fleshed out, in particular more descriptive markdown sections describing the functionality being demonstrated.
+
+Documentation
+~~~~~~~~~~~~~
+
+* Docstrings were fixed for some functions that were improperly formatted.
+
+* Docstrings were updated for `pvops.text.visualize` functions' new `ax` argument.
+
+Other
+~~~~~~
+
+* Minor changes to plotting functions in the submodule `pvops.text.visualize` related to text size and other miscellaneous aspects of figures.
+
+* Python versions for testing/linting changed from 3.8-3.11 to 3.9-3.13. 3.8 has reached end-of-life, and 3.13 is current (see: https://devguide.python.org/versions/).
+
+* Building docs changed to use Python 3.13 (previously 3.11).

docs/pages/tutorials.rst

@@ -1,20 +1,26 @@
 pvOps Tutorials
 ===============
 
-Check out the tutorials below!
+Check out the tutorials below (organized by module)!
 
 .. nbgallery::
-   :caption: Text & Text2Time tutorials:
+   :caption: Text tutorials:
 
-   tutorials/tutorial_text2time_module
-   tutorials/tutorial_textmodule
+   tutorials/tutorial_text_1_preprocessing
+   tutorials/tutorial_text_2_keyword_classification
+   tutorials/tutorial_text_3_ml_classification
+
+.. nbgallery::
+   :caption: Text2Time tutorial:
+
+   tutorials/tutorial_text2time
 
 .. nbgallery::
    :caption: Timeseries tutorials:
 
-   tutorials/tutorial_timeseries
-   tutorials/tutorial_AIT_timeseries
-   tutorials/tutorial_timeseries_sim
+   tutorials/tutorial_timeseries_energy_modeling
+   tutorials/tutorial_timeseries_AIT
+   tutorials/tutorial_timeseries_simulation
    tutorials/tutorial_timeseries_survival_analysis
 
 .. nbgallery::

docs/pages/tutorials/tutorial_text2time.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_text2time.ipynb"
+}

docs/pages/tutorials/tutorial_text_1_preprocessing.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_text_1_preprocessing.ipynb"
+}

docs/pages/tutorials/tutorial_text_2_keyword_classification.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_text_2_keyword_classification.ipynb"
+}

docs/pages/tutorials/tutorial_text_3_ml_classification.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_text_3_ml_classification.ipynb"
+}

docs/pages/tutorials/tutorial_timeseries_AIT.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_timeseries_AIT.ipynb"
+}

docs/pages/tutorials/tutorial_timeseries_energy_modeling.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_timeseries_energy_modeling.ipynb"
+}

docs/pages/tutorials/tutorial_timeseries_simulation.nblink

@@ -0,0 +1,3 @@
+{
+    "path": "../../../tutorials/tutorial_timeseries_simulation.ipynb"
+}

docs/pages/userguide.rst

@@ -12,10 +12,10 @@ User Guide
     :maxdepth: 1
     :caption: Module Guides
 
-    moduleguides/text
-    moduleguides/text2time
-    moduleguides/timeseries
-    moduleguides/iv
+    moduleguides/guide_text
+    moduleguides/guide_text2time
+    moduleguides/guide_timeseries
+    moduleguides/guide_iv
 
 .. toctree::
     :maxdepth: 1

pvops/__init__.py

@@ -8,7 +8,7 @@
     # warnings.warn("")
     pass
 
-__version__ = '0.6.1'
+__version__ = '0.7.0'
 
 __copyright__ = """Copyright 2023 National Technology & Engineering 
 Solutions of Sandia, LLC (NTESS). Under the terms of Contract DE-NA0003525 

pvops/iv/extractor.py

@@ -133,7 +133,7 @@ def create_string_object(self, iph, io, rs, rsh, nnsvth):
 
         if self.verbose >= 2:
             print(
-                f'\tSimulations completed after {round(time.time()-start_t,2)} seconds')
+                f'\tSimulations completed after {round(time.time() - start_t, 2)} seconds')
 
         return sim
 

pvops/iv/simulator.py

@@ -823,13 +823,13 @@ def BISHOP88_simulate_module(self, mod_key):
 
                     substr_v = bypass(
                         substr_v, self.module_parameters['v_bypass'])
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['V'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['V'].append(
                         substr_v)
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['I'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['I'].append(
                         substr_i)
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['T'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['T'].append(
                         sum(temps) / len(temps))
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['E'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['E'].append(
                         sum(irrs) / len(irrs))
 
                     mod_v, mod_i = add_series(substr_v, substr_i, mod_v, mod_i)
@@ -986,9 +986,9 @@ def find_nearest(array, value):
 
                     substr_v = bypass(
                         substr_v, self.module_parameters['v_bypass'])
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['V'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['V'].append(
                         substr_v)
-                    self.multilevel_ivdata['module'][mod_key][f'substr{s+1}']['I'].append(
+                    self.multilevel_ivdata['module'][mod_key][f'substr{s + 1}']['I'].append(
                         substr_i)
 
                     if show_debugging_plots:
@@ -1714,7 +1714,7 @@ def visualize_cell_level_traces(self, cell_identifier, cutoff=True, table=True,
             axs[1].axis('off')
 
             rowlabels = [
-                f'condition{i+1}' for i in range(len(formatted_conds))]
+                f'condition{i + 1}' for i in range(len(formatted_conds))]
             our_colors = sns.color_palette("hls", len(rowlabels))
             collabels = self.acceptible_keys
 

pvops/text/classify.py

@@ -21,11 +21,12 @@ def classification_deployer(
     greater_is_better=True,
     verbose=3,
 ):
-    """The classification deployer builds a classifier evaluator with an ingrained hyperparameter fine-tuning grid search protocol.
+    """
+    The classification deployer builds a classifier evaluator with an ingrained hyperparameter fine-tuning grid search protocol.
     The output of this function will be a data frame showing the performance of each classifier when utilizing a specific hyperparameter
     configuration.
 
-    To see an example of this method's application, see ``tutorials//text_class_example.py``
+    To see an example of this method's application, see ``tutorials/tutorial_text_preprocess_classify.ipynb``.
 
     Parameters
     ----------
@@ -190,7 +191,8 @@ def classification_deployer(
     return pd.concat(rows, axis=1).T, best_gs_instance.best_estimator_
 
 def get_attributes_from_keywords(om_df, col_dict, reference_df, reference_col_dict):
-    """Find keywords of interest in specified column of dataframe, return as new column value.
+    """
+    Find keywords of interest in specified column of dataframe, return as new column value.
 
     If keywords of interest given in a reference dataframe are in the specified column of the
     dataframe, return the keyword category, or categories.
@@ -200,25 +202,22 @@ def get_attributes_from_keywords(om_df, col_dict, reference_df, reference_col_di
     ----------
     om_df : pd.DataFrame
         Dataframe to search for keywords of interest, must include text_col.
-    col_dict : dict of {str : str}
+    col_dict : dict of {str: str}
         A dictionary that contains the column names needed:
-
         - data : string, should be assigned to associated column which stores the tokenized text logs
         - predicted_col : string, will be used to create keyword search label column
     reference_df : DataFrame
         Holds columns that define the reference dictionary to search for keywords of interest,
         Note: This function can currently only handle single words, no n-gram functionality.
-    reference_col_dict : dict of {str : str}
+    reference_col_dict : dict of {str: str}
         A dictionary that contains the column names that describes how
         referencing is going to be done
-
         - reference_col_from : string, should be assigned to
-          associated column name in reference_df that are possible input reference values
-          Example: pd.Series(['inverter', 'invert', 'inv'])
+        associated column name in reference_df that are possible input reference values.
+        Example: pd.Series(['inverter', 'invert', 'inv'])
         - reference_col_to : string, should be assigned to
-          associated column name in reference_df that are the output reference values
-          of interest
-          Example: pd.Series(['inverter', 'inverter', 'inverter'])
+        associated column name in reference_df that are the output reference values of interest.
+        Example: pd.Series(['inverter', 'inverter', 'inverter'])
 
     Returns
     -------