DOC #166

CHANGES.rst

@@ -7,8 +7,13 @@ Change History
 Production
 **********
 
-:2020.0.2: *2019.07.09*, bug fixes and code review suggestions
+:2021.0.0: *2019.07.12*, API change affecting plugins
 
+    API change:
+    Changed how plugins are defined and registered.
+
+    * `#168 <https://github.com/prjemian/spec2nexus/pull/168>`_
+       plugins are now self-registering
     * `#166 <https://github.com/prjemian/spec2nexus/issues/166>`_
        fix conda packaging
 

docs/source/how_to_write_plugin.rst

@@ -4,23 +4,28 @@
 How to write a custom plugin module
 ###################################
 
+.. warning::  This documentation must be revised for releases 2021.0.0 and beyond.
+
+   It describes how to write plugins up to release 2020.0.2.
+
+   * SAME: The basics of writing the plugins remains the same.
+   * CHANGED: The method of registering the plugins has changed.
+   * CHANGED: The declaration of each plugin has changed.
+   * CHANGED: The name of each plugin file has been relaxed.
+   * CHANGED: Plugin files do not have to be in their own directory.
+   * REMOVED: The ``SPEC2NEXUS_PLUGIN_PATH`` environment variable has been eliminated.
+
 A custom plugin module for :mod:`spec2nexus.spec` is provided in a python module (Python source code file).
 In this custom plugin module are subclasses for each *new* control line to be supported.  An exception will 
 be raised if a custom plugin module tries to provide support for an existing control line.  
 
-Give the custom plugin module a name ending with ``_spec2nexus.py``.
-Ensure this name is different than any other plugin module you will use
-(currently, avoid ``spec_common_spec2nexus.py``, ``uim_spec2nexus.py``, 
-and ``unicat_spec2nexus.py``) to avoid possible duplication.
-
-The custom plugin module can be stored in any directory that is convenient.
-Define the environment variable ``SPEC2NEXUS_PLUGIN_PATH`` with the
-directory (directories, comma delimited) where your plugin file(s) reside.
-On linux, with the bash shell, this might be::
-
-    export SPEC2NEXUS_PLUGIN_PATH="/home/jemian/.spec2nexus_plugins, /tmp"
+Give the custom plugin module a name ending with ``.py``.
+As with any Python module, the name must be unique within a directory.
+There must be a ``__init__.py`` file in the same directory (even if 
+that file is empty) so that your plugin module can be loaded with ``import <MODULE>``.
 
-The custom plugin module should contain, at minimum one subclass of  
+Please view the existing plugins in :mod:`~spec2nexus.plugins.spec_common`
+for examples.  The custom plugin module should contain, at minimum one subclass of  
 :class:`spec2nexus.plugin.ControlLineHandler`.  A custom plugin module
 can contain many such handlers, as needs dictate.
 
@@ -53,6 +58,9 @@ the function that matches for a particular control line key.  Override the
 handler's :meth:`match_key` method.
 For more details, see the section :ref:`custom_key_match_function`.
 
+.. note:: The ``six`` package is used to make our plugins run with either
+   Python 2.7 or Python 3.5+.
+
 Example for **#U** control line
 *******************************
 
@@ -92,8 +100,10 @@ Consider that one SPEC file contains the control line: ``#Y 1 2 3 4 5``.
 Since there is no standard handler for this control line, we create one that
 ignores processing by doing nothing::
 
-   from spec2nexus.plugin import ControlLineHandler
+   import six
+   from spec2nexus.plugin import AutoRegister, ControlLineHandler
    
+   @six.add_metaclass(AutoRegister)
    class Ignore_Y_ControlLine(ControlLineHandler):
        '''**#Y** -- as in ``#Y 1 2 3 4 5``'''
    
@@ -115,7 +125,7 @@ The postprocessing method is registered from the control line handler by calling
 :meth:`addPostProcessor` method of the ``spec_obj`` argument received by the 
 handler's :meth:`process` method.  A key name [#]_ is supplied when registering to avoid 
 registering this same code more than once.  The postprocessing function will be called 
-with the instance of :class:`spec2nexus.spec.SpecDataFileScan` as its only argument.
+with the instance of :class:`~spec2nexus.spec.SpecDataFileScan` as its only argument.
 
 An important role of the postprocessing is to store the result in the scan object.
 It is important not to modify other data in the scan object.  Pick an attribute
@@ -157,8 +167,11 @@ Summary Example Custom Plugin with postprocessing
 
 Gathering all parts of the examples above, the custom plugin module is::
 
-   from spec2nexus.plugin import ControlLineHandler, strip_first_word
+   import six
+   from spec2nexus.plugin import strip_first_word
+   from spec2nexus.plugin import AutoRegister, ControlLineHandler
    
+   @six.add_metaclass(AutoRegister)
    class User_ControlLine(ControlLineHandler):
        '''**#U** -- User data (#U user1 user2 user3)'''
    
@@ -304,23 +317,23 @@ can override the :meth:`match_key()` method.  Here is an example::
 Summary Requirements for custom plugin
 **************************************
 
-* file name must end in ``_spec2nexus.py``
-* file can go in any directory
-* add directory to ``SPEC2NEXUS_PLUGIN_PATH`` environment variable (comma-delimited for multiple directories)
+* file can go in any directory that has ``__init__.py`` file
 * multiple control line handlers can go in a single file
 * for each control line:
 
   * subclass :class:`spec2nexus.plugin.ControlLineHandler`
+  * add ``@six.add_metaclass(AutoRegister)`` decorator to auto-register the plugin
+  * import the module you defined (FIXME:  check this and revise)
   * identify the control line pattern
   * define ``key`` with a regular expression to match [#]_
 
     * ``key`` is used to identify control line handlers
     * redefine existing supported control lines to replace supplied behavior (use caution!)
     * Note: ``key="scan data"`` is used to process the scan data: :meth:`spec2nexus.plugins.spec_common_spec2nexus.SPEC_DataLine`
 
-  * (optional) define :meth:`match_key` to override the default regular expression to match the key
   * define :meth:`process` to handle the supplied text
   * define :meth:`writer` to write the in-memory data structure from this plugin to HDF5+NeXus data file
+  * (optional) define :meth:`match_key` to override the default regular expression to match the key
 
 * for each postprocessing function:
 
@@ -329,5 +342,5 @@ Summary Requirements for custom plugin
 
 .. [#] It is possible to override the default regular expression match
    in the subclass with a custom match function.  See the
-   :meth:`spec2nexus.plugins.spec_common_spec2nexus.SPEC_DataLine.match_key()`
+   :meth:`~spec2nexus.plugins.spec_common.SPEC_DataLine.match_key()`
    method for an example.

docs/source/plugin.rst

@@ -13,7 +13,8 @@ provided in the scan header.
 
 Plugins are now used to handle all control lines in :mod:`spec2nexus.spec`.
 Any control line encountered but not recognized will be placed as text
-in a NeXus **NXnote** group named ``unrecognized``.
+in a NeXus **NXnote** group named ``unrecognized_NNN`` (where ``NNN``
+is from 1 to the maximum number of unrecognized control lines).
 
 .. _supplied_plugins:
 
@@ -25,11 +26,11 @@ These plugin modules are supplied:
 .. autosummary::
    :nosignatures:
 
-   ~spec2nexus.plugins.spec_common_spec2nexus
-   ~spec2nexus.plugins.fallback_spec2nexus
-   ~spec2nexus.plugins.unicat_spec2nexus
-   ~spec2nexus.plugins.uim_spec2nexus
-   ~spec2nexus.plugins.XPCS_spec2nexus
+   ~spec2nexus.plugins.spec_common
+   ~spec2nexus.plugins.fallback
+   ~spec2nexus.plugins.unicat
+   ~spec2nexus.plugins.uim
+   ~spec2nexus.plugins.XPCS
 
 .. toctree::
    :maxdepth: 2
@@ -62,39 +63,39 @@ Plugins for these control lines [#]_ are provided in **spec2nexus**:
 .. autosummary::
    :nosignatures:
 
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_File
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Epoch
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Date
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Comment
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Geometry
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_NormalizingFactor
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_CounterNames
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_CounterMnemonics
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Labels
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Monitor
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_NumColumns
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_PositionerNames
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_PositionerMnemonics
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Positioners
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_HKL
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_Scan
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_CountTime
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_UserReserved
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_TemperatureSetPoint
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_DataLine
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_Array
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_Calibration
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_ChannelInformation
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_CountTime
-   ~spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_RegionOfInterest
-   ~spec2nexus.plugins.fallback_spec2nexus.UnrecognizedControlLine
-   ~spec2nexus.plugins.unicat_spec2nexus.UNICAT_MetadataMnemonics
-   ~spec2nexus.plugins.unicat_spec2nexus.UNICAT_MetadataValues
-   ~spec2nexus.plugins.uim_spec2nexus.UIM_generic
-   ~spec2nexus.plugins.XPCS_spec2nexus.XPCS_VA
-   ~spec2nexus.plugins.XPCS_spec2nexus.XPCS_VD
-   ~spec2nexus.plugins.XPCS_spec2nexus.XPCS_VE
+   ~spec2nexus.plugins.spec_common.SPEC_File
+   ~spec2nexus.plugins.spec_common.SPEC_Epoch
+   ~spec2nexus.plugins.spec_common.SPEC_Date
+   ~spec2nexus.plugins.spec_common.SPEC_Comment
+   ~spec2nexus.plugins.spec_common.SPEC_Geometry
+   ~spec2nexus.plugins.spec_common.SPEC_NormalizingFactor
+   ~spec2nexus.plugins.spec_common.SPEC_CounterNames
+   ~spec2nexus.plugins.spec_common.SPEC_CounterMnemonics
+   ~spec2nexus.plugins.spec_common.SPEC_Labels
+   ~spec2nexus.plugins.spec_common.SPEC_Monitor
+   ~spec2nexus.plugins.spec_common.SPEC_NumColumns
+   ~spec2nexus.plugins.spec_common.SPEC_PositionerNames
+   ~spec2nexus.plugins.spec_common.SPEC_PositionerMnemonics
+   ~spec2nexus.plugins.spec_common.SPEC_Positioners
+   ~spec2nexus.plugins.spec_common.SPEC_HKL
+   ~spec2nexus.plugins.spec_common.SPEC_Scan
+   ~spec2nexus.plugins.spec_common.SPEC_CountTime
+   ~spec2nexus.plugins.spec_common.SPEC_UserReserved
+   ~spec2nexus.plugins.spec_common.SPEC_TemperatureSetPoint
+   ~spec2nexus.plugins.spec_common.SPEC_DataLine
+   ~spec2nexus.plugins.spec_common.SPEC_MCA
+   ~spec2nexus.plugins.spec_common.SPEC_MCA_Array
+   ~spec2nexus.plugins.spec_common.SPEC_MCA_Calibration
+   ~spec2nexus.plugins.spec_common.SPEC_MCA_ChannelInformation
+   ~spec2nexus.plugins.spec_common.SPEC_MCA_CountTime
+   ~spec2nexus.plugins.spec_common.SPEC_MCA_RegionOfInterest
+   ~spec2nexus.plugins.fallback.UnrecognizedControlLine
+   ~spec2nexus.plugins.unicat.UNICAT_MetadataMnemonics
+   ~spec2nexus.plugins.unicat.UNICAT_MetadataValues
+   ~spec2nexus.plugins.uim.UIM_generic
+   ~spec2nexus.plugins.XPCS.XPCS_VA
+   ~spec2nexus.plugins.XPCS.XPCS_VD
+   ~spec2nexus.plugins.XPCS.XPCS_VE
 
 .. [#] Compare this list with :ref:`plugin_list`
 

docs/source/spec.rst

@@ -317,7 +317,7 @@ the next line is read as part of this line.)
 
 This is an example of a 91-channel MCA data array with trivial (zero) values:
 
-.. code-block:: guess
+.. code-block:: text
    :linenos:
    
    @A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\
@@ -331,7 +331,7 @@ Several MCA spectra may be written to a scan.  In this case, a number
 follows **@A** indicating which spectrum, such as in this example with
 four spectra:
 
-.. code-block:: guess
+.. code-block:: text
    :linenos:
 
 	@A1 0 0 0 0 0 0 35 0 0 35

docs/source/supplied_plugins/XPCS_spec2nexus.rst

@@ -1,9 +1,9 @@
 
-.. _XPCS_spec2nexus:
+.. _XPCS:
 
 XPCS plugin
 ###########
 
-.. automodule:: spec2nexus.plugins.XPCS_spec2nexus
+.. automodule:: spec2nexus.plugins.XPCS
     :members: 
     :synopsis: SPEC data file control lines unique to the APS XPCS instrument

docs/source/supplied_plugins/fallback_spec2nexus.rst

@@ -1,9 +1,9 @@
 
-.. _fallback_spec2nexus:
+.. _fallback:
 
 Fallback plugin
 ###############
 
-.. automodule:: spec2nexus.plugins.fallback_spec2nexus
+.. automodule:: spec2nexus.plugins.fallback
     :members: 
     :synopsis: Fallback handling for any SPEC data file control lines not recognized by other handlers.

docs/source/supplied_plugins/spec_common_spec2nexus.rst

@@ -4,6 +4,6 @@
 SPEC standard plugin
 ####################
 
-.. automodule:: spec2nexus.plugins.spec_common_spec2nexus
+.. automodule:: spec2nexus.plugins.spec_common
     :members: 
     :synopsis: SPEC standard data file support.

docs/source/supplied_plugins/uim_spec2nexus.rst

@@ -4,6 +4,6 @@
 UIM plugin
 ##########
 
-.. automodule:: spec2nexus.plugins.uim_spec2nexus
+.. automodule:: spec2nexus.plugins.uim
     :members: 
     :synopsis: UIM - Image header information from EPICS areaDetector.

docs/source/supplied_plugins/unicat_spec2nexus.rst

@@ -4,6 +4,6 @@
 UNICAT metadata plugin
 ######################
 
-.. automodule:: spec2nexus.plugins.unicat_spec2nexus
+.. automodule:: spec2nexus.plugins.unicat
     :members: 
     :synopsis: Metadata in SPEC data files as defined by APS UNICAT.

src/spec2nexus/__init__.py

@@ -35,6 +35,7 @@
     'matplotlib', 
     'matplotlib.backends', 
     'matplotlib.backends.backend_agg',
+    'matplotlib.figure', 
     # do NOT mock the "six" package!
 ]
 __classifiers__ = [

src/spec2nexus/plugin.py

@@ -161,7 +161,7 @@ class AutoRegister(type):
     
     This class is a metaclass to auto-register plugins to handle
     various parts of a SPEC data file.  
-    See :module:`~spec2nexus.plugins.spec_common` for many examples.
+    See :mod:`~spec2nexus.plugins.spec_common` for many examples.
 
     :param str key: regular expression to match a control line key, up to the first space
     :returns: None