The Big Merge

.gitignore

@@ -3,7 +3,7 @@
 development_scripts/*/*.png
 development_scripts/*/*.svg
 development_scripts/*/*.csv
-docs/source/figures/big_src/*
+docs/source/figures/big_src/
 
 
 

.readthedocs.yml

@@ -18,8 +18,7 @@ sphinx:
 #   - pdf
 
 # Optionally set the version of Python and requirements required to build your docs
-# python:
-#  version: 3.7
-#  install:
-#    - requirements: docs/requirements.txt
-#    - requirements: docs/requirements.txt
+python:
+  version: 3.6
+  install:
+    - requirements: docs/requirements.txt

README.rst

@@ -1,18 +1,128 @@
+.. figure:: docs/source/figures/logo.svg
+    :width: 200
+
 =============================================
 ``ixdat``: The In-situ Experimental Data Tool
 =============================================
 
-NOTE: We are currently working mainly on the `[user_ready] <https://github.com/ixdat/ixdat/tree/user_ready>`_ branch. Both the `documentation <https://ixdat.readthedocs.io>`_ and the `pip installation <https://pypi.org/project/ixdat/>`_ are presently compiled from that branch while we work through some important details in `a pull request <https://github.com/ixdat/ixdat/pull/6>`_. 
+With ``ixdat``, you can import, combine, and export complex experimental datasets
+as simply as::
+
+    ec = Measurement.read_set("awesome_EC_data", reader="biologic")
+    ec.plot_measurement()
+
+    ms = Measurement.read("2021-03-30 16_59_35 MS data.tsv", reader="zilien")
+    ms.plot_measurement()
+
+    ecms = ec + ms
+    ecms.plot_measurement()
+
+    ecms.export("my_combined_data.csv")
+
+Output:
+
+.. figure:: docs/source/figures/ixdat_example_figures.png
+    :width: 700
+
+    In-situ experimental data made easy
+
+Or rather than exporting, you can take advantage of ``ixdat``'s powerful analysis
+tools and database backends to be a one-stop tool from messy raw data to public
+repository accompanying your breakthrough publication and advancing our field.
+
+About
+-----
+
+``ixdat`` provides a powerful **object-oriented** interface to experimental data, especially in-situ experimental data for which it is of interest to combine data obtained simultaneously from multiple techniques.
+
+Documentation is at https://ixdat.readthedocs.io
+
+In addition to a **pluggable** parser interface for importing your data format, ``ixdat`` also includes
+pluggable exporters and plotters, as well as a database interface. A relational model of experimental data is
+designed into every level.
+
+.. list-table:: Techniques and Readers
+   :widths: 20 15 50
+   :header-rows: 1
+
+
+   * - Measurement technique
+     - Status
+     - Readers
+   * - Electrochemistry
+     - Released
+     - - biologic: .mpt files from Biologic's EC-Lab software
+       - autolab: ascii files from AutoLab's NOVA software
+       - ivium: .txt files from Ivium's IviumSoft software
+   * - Mass Spectrometry
+     - Released
+     - - pfeiffer: .dat files from Pfeiffer Vacuum's PVMassSpec software
+       - cinfdata: text export from DTU Physics' cinfdata system
+       - zilien: .tsv files from Spectro Inlets' Zilien software
+   * - Electrochemistry - Mass Spectrometry
+     - Released
+     - - zilien: .tsv files from Spectro Inlets' Zilien software
+       - EC_MS: .pkl files from the legacy EC_MS python package
+   * - Spectroelectrochemistry
+     - Released
+     - - msrh_sec: .csv file sets from Imperial College London's SEC system
+   * - X-ray photoelectron spectroscopy (XPS)
+     - Future
+     -
+   * - X-ray diffraction (XRD)
+     - Future
+     -
+   * - Low-Energy Ion Scattering (LEIS)
+     - Future
+     -
+
+Tutorials are described at https://ixdat.readthedocs.io/en/latest/tutorials.html
+
+Installation
+------------
+
+To use ``ixdat``, you need to have python installed. We recommend
+`Anaconda python <https://www.anaconda.com/products/individual>`_.
+
+To install ``ixdat``, just type in your terminal or Anaconda prompt::
+
+    $ pip install ixdat
+
+And hit enter.
+
+``ixdat`` is under development, and to make use of the newest features,
+you may need to upgrade to the latest version. This is also easy. Just type::
+
+    $ pip install --upgrade ixdat
+
+
+Article repositories
+--------------------
+
+``ixdat`` is shown in practice in a growing number of open repositories of data and analysis
+for academic publications:
+
+- Soren B. Scott, et al.  **Tracking oxygen atoms in electrochemical CO oxidation –Part I: Oxygen exchange via CO2 hydration**. `Electrochimica Acta, 374, 137842 <https://doi.org/10.1016/j.electacta.2021.137842>`_, **2021**.
+
+  Repository: https://github.com/ScottSoren/pyCOox_public
+
+- Soren B. Scott, et al.  **Tracking oxygen atoms in electrochemical CO oxidation –Part II: Lattice oxygen reactivity in oxides of Pt and Ir**. `Electrochimica Acta, 374, 137844 <https://doi.org/10.1016/j.electacta.2021.137844>`_, **2021**.
+
+  Repository: https://github.com/ScottSoren/pyCOox_public
+
+- Kevin Krempl, et al. **Dynamic Interfacial Reaction Rates from Electrochemistry - Mass Spectrometry**. `Journal of Analytical Chemistry. 93, 7022-7028 <https://doi.org/10.1021/acs.analchem.1c00110>`_, **2021**
+
+  Repository: https://github.com/kkrempl/Dynamic-Interfacial-Reaction-Rates
 
-``ixdat`` will provide a powerful **object-oriented** interface to experimental data, especially in-situ experimental data for which it is of interest to combine data obtained simultaneously from multiple techniques.
+- Junheng Huang, et al. **Online Electrochemistry−Mass Spectrometry Evaluation of the Acidic Oxygen Evolution Reaction at Supported Catalysts**. `ACS Catal. 11, 12745-12753 <https://doi.org/10.1021/acscatal.1c03430>`_, **2021**
 
-``ixdat`` will replace the existing electrochemistry - mass spectrometry data tool, `EC_MS <https://github.com/ScottSoren/EC_MS>`_, and will thus become a powerful stand-alone tool for analysis and visualization of data acquired by the equipment of `Spectro Inlets <https://spectroinlets.com>`_ and other EC-MS solutions.
-It will also replace the existing electrochemistry - synchrotron GIXRD data tool, `EC_Xray <https://github.com/ScottSoren/EC_Xray>`_ when needed.
-Over time, it will acquire functionality for more and more techniques.
+  Repository: https://github.com/ScottSoren/Huang2021
 
-In addition to a **pluggable** parser interface for importing your data format, it will include pluggable exporters and plotters, as well as a database interface.
 
-We will update this README as features are added. More importantly, we will document this project as we develop it here: https://ixdat.readthedocs.io/
+Join us
+-------
 
-``ixdat`` is free and open source software and we welcome input and new collaborators.
+`ixdat`` is free and open source software and we welcome input and new collaborators. Please help us improve!
 
+Contact us (sbscott@ic.ac.uk) or just
+`get started developing <https://ixdat.readthedocs.io/en/latest/developing.html>`_.

TOOLS.rst

@@ -12,7 +12,7 @@ development:
 * **sphinx** is used to build documentation
 
 The following is a list of "tool and commands runners":
-   
+
 * **invoke** is used during development, to run tools and other
   pre-configured maintenance tasks inside the existing development
   environment
@@ -42,6 +42,8 @@ summary here with our suggestions for usage.
 
 Tools
 -----
+black
+.....
 **black** is an autoformatter, which fixes your white space usage etc.
 https://black.readthedocs.io/en/stable/
 It's nice to run it from the same terminal
@@ -50,6 +52,8 @@ before committing (and avoid a "fix formatting" commit later). To get black
 and the other tools available in git bash, you have to tell ~\.bashrc where
 it is (see Windows instructions below under git hooks).
 
+flake8
+......
 **flake8** is a linter, which checks the code for errors.
 https://flake8.pycqa.org/en/latest/
 This includes
@@ -64,9 +68,29 @@ Additional allowances may need to be added there.
 flake8 also enforces a maximum line length of 89, chosen to match the
 default setting of black (+/- 1 char).
 
+pytest
+......
 **pytest** is a suite of stuff used to write and run software tests.
 https://docs.pytest.org/en/stable/
 
+sphinx
+......
+**sphinx** is used to generate the beautiful documentation on
+https://ixdat.readthedocs.io from ReStructuredText and ixdat source code.
+To set it up just install sphinx, if you haven't already. In your terminal or Anaconda prompt, type::
+
+  $ pip install sphinx
+
+Then, to build the documentation, navigate to ``ixdat/docs``, and run in your terminal or Anaconda prompt::
+
+  $ ./make html
+
+Note, if you get an "access denied" error, you will just need to run the terminal as an administrator.
+
+Then you can see the built documentation in your browser by double-clicking
+``ixdat/docs/build/html/index.html``
+
+
 **sphinx** is a tool for building the documentation into html and other
 formats from restructured text (.rst). It also enables automatic documentation
 generation from the doc-strings in the code. Read The Docs uses sphinx to compile

development_scripts/append_ec_files.py

@@ -74,7 +74,7 @@
 cv_selection = cv[10:16]
 
 cv_selection.plot_measurement(j_name="cycle")
-cv_selection.redefine_cycle(start_potential=0.4, redox=1)
+cv_selection.redefine_cycle(start_potential=0.45, redox=1)
 cv_selection.plot_measurement(j_name="cycle")
 
 ax = cv_selection[1].plot(label="cycle 1")

development_scripts/functional_test.py

@@ -28,20 +28,18 @@
 
 assert np.isclose(
     ec_measurement.v[0] - ec_measurement["raw_potential"].data[0],
-    ec_measurement.RE_vs_RHE
+    ec_measurement.RE_vs_RHE,
 )
 
 # To make it complex, we first select a couple cycles, this time by converting
 # to a cyclic voltammagram and indexing, and append them:
 cv = ec_measurement.as_cv()
 cvs_1_plus_2 = cv[1] + cv[2]
 
-# Check that the calibration survived all that:
+# Check that the ms_calibration survived all that:
 assert cvs_1_plus_2.RE_vs_RHE == ec_measurement.RE_vs_RHE
 # Check that the main time variable, that of potential, wasn't corrupted:
-assert len(cvs_1_plus_2.grab("potential")[0]) == len(
-    cvs_1_plus_2["time/s"].data
-)
+assert len(cvs_1_plus_2.grab("potential")[0]) == len(cvs_1_plus_2["time/s"].data)
 # Check that the selector is still available and works with the main time var:
 assert len(cvs_1_plus_2.selector.data) == len(cvs_1_plus_2.t)
 

development_scripts/reader_testers/test_autolab_reader.py

@@ -0,0 +1,14 @@
+"""For use in development of the autolab reader. Requires access to sample data."""
+
+from pathlib import Path
+from matplotlib import pyplot as plt
+
+from ixdat import Measurement
+
+path_to_file = Path.home() / (
+    "Dropbox/ixdat_resources/test_data/autolab/autolab_test_file.txt"
+)
+
+meas = Measurement.read(path_to_file, reader="autolab")
+
+meas.plot()

development_scripts/reader_testers/test_cinfdata_reader.py

@@ -0,0 +1,45 @@
+"""For use in development of the cinfdata reader. Requires access to sample data."""
+
+from pathlib import Path
+from matplotlib import pyplot as plt
+
+from ixdat import Measurement
+
+path_to_file = (
+    Path.home()
+    / "Dropbox/ixdat_resources/test_data/cinfdata/Trimarco2018_fig3/QMS_1.txt"
+)
+ms_meas = Measurement.read(path_to_file, reader="cinfdata")
+ms_meas.plot_measurement()
+
+path_to_ec_file_start = (
+    Path.home() / "Dropbox/ixdat_resources/test_data/cinfdata/Trimarco2018_fig3/09_fig4"
+)
+ec_meas = Measurement.read_set(path_to_ec_file_start, reader="biologic")
+ec_meas.calibrate(RE_vs_RHE=0.65, A_el=0.196)
+ec_meas.plot_measurement()
+
+ecms_meas = ec_meas + ms_meas
+axes = ecms_meas.plot_measurement(
+    mass_lists=[["M44", "M2"], ["M4", "M28"]],
+    tspan_bg=[30, 40],
+    legend=False,
+    unit="pA",
+)
+
+axes[0].set_ylim([-7, 70])
+axes[2].set_ylim([-1.8e3, 18e3])
+fig = axes[0].get_figure()
+fig.tight_layout()
+# fig.savefig("../../docs/source/figures/ec_ms.svg")
+
+ecms_meas.set_bg(tspan_bg=[0, 10])
+
+cv = ecms_meas.as_cv()
+cv.redefine_cycle(start_potential=0.39, redox=False)
+
+axes_cv = cv[2].plot(mass_list=["M2", "M44"], logplot=False, )
+axes_cv = cv[1].plot(
+    mass_list=["M2", "M44"], linestyle="--", axes=axes_cv, logplot=False,
+)
+axes_cv[0].get_figure().savefig("Trimarco2018_ixdat.png")

development_scripts/reader_testers/test_ivium_reader.py

@@ -0,0 +1,28 @@
+"""For use in development of the ivium reader. Requires access to sample data."""
+
+from pathlib import Path
+import pandas as pd
+
+from ixdat import Measurement
+from ixdat.techniques import CyclicVoltammogram
+
+path_to_file = Path.home() / (
+    "Dropbox/ixdat_resources/test_data/ivium/ivium_test_dataset"
+)
+path_to_single_file = path_to_file.parent / (path_to_file.name + "_1")
+df = pd.read_csv(path_to_single_file, sep=r"\s+", header=1)
+
+meas = Measurement.read(path_to_file, reader="ivium")
+
+meas.save()
+
+meas.plot_measurement()
+
+meas_cv = CyclicVoltammogram.read(path_to_file, reader="ivium")
+
+meas_cv.save()
+
+meas_cv.plot_measurement()
+meas_cv.redefine_cycle(start_potential=0.4, redox=False)
+for i in range(4):
+    meas_cv[i].plot()

development_scripts/reader_testers/test_ixdat_csv_reader.py

@@ -0,0 +1,31 @@
+"""For use in development of the ixdat .csv reader (and exporter)."""
+
+from ixdat import Measurement
+
+
+if False:  # test the version that's online on the tutorials page
+    meas = Measurement.read_url(
+        "https://raw.githubusercontent.com/ixdat/tutorials/"
+        + "main/loading_appending_and_saving/co_strip.csv",
+        reader="ixdat",
+    )
+    meas.plot_measurement()
+
+else:
+    meas = Measurement.read(
+        "../../test_data/biologic/Pt_poly_cv_CUT.mpt", reader="biologic"
+    )
+    meas.calibrate_RE(0.715)
+
+    meas.correct_ohmic_drop(R_Ohm=100)
+
+    meas.normalize_current(0.196)
+
+    cv = meas.as_cv()
+
+    cv.export("test.csv")
+
+    meas_loaded = Measurement.read("test.csv", reader="ixdat")
+
+    meas_loaded.plot()
+