Support python 3.12 / notebook>=7 and drop support for python 3.9.

.github/workflows/pylake_docs_test.yml

@@ -9,13 +9,13 @@ jobs:
     build_docs:
         runs-on: ubuntu-latest
         steps:
-        - uses: actions/checkout@v3
+        - uses: actions/checkout@v4
           with:
             lfs: true
         - name: Set up Python
-          uses: actions/setup-python@v4
+          uses: actions/setup-python@v5
           with:
-              python-version: '3.9'
+              python-version: '3.10'
               cache: 'pip'
               cache-dependency-path: setup.py
         - name: Install dependencies

.github/workflows/pylake_release.yml

@@ -13,13 +13,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
         lfs: true
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.10'
         cache: 'pip'
         cache-dependency-path: setup.py
     - name: Install dependencies

.github/workflows/pylake_test.yml

@@ -12,14 +12,14 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, windows-latest, macos-latest]
-        python-version: ["3.9", "3.11"]
+        python-version: ["3.10", "3.12"]
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
         lfs: true
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
         cache: 'pip'

.github/workflows/pylake_test_no_notebook.yml

@@ -9,13 +9,13 @@ jobs:
     build_no_nb:
         runs-on: ubuntu-latest
         steps:
-            - uses: actions/checkout@v3
+            - uses: actions/checkout@v4
               with:
                   lfs: true
             - name: Set up Python
-              uses: actions/setup-python@v4
+              uses: actions/setup-python@v5
               with:
-                  python-version: '3.9'
+                  python-version: '3.10'
                   cache: 'pip'
                   cache-dependency-path: setup.py
             - name: Install dependencies

.readthedocs.yaml

@@ -3,7 +3,7 @@ version: 2
 build:
   os: ubuntu-22.04
   tools:
-    python: "3.9"
+    python: "3.10"
 
 python:
   install:

changelog.md

@@ -4,6 +4,7 @@
 
 #### New features
 
+* Support Python `3.12`, dropped support for `3.9`.
 * Added [`Kymo.plot_with_channels()`](https://lumicks-pylake.readthedocs.io/en/latest/_api/lumicks.pylake.kymo.Kymo.html#lumicks.pylake.kymo.Kymo.plot_with_channels) for plotting a kymograph with corresponding channel data. For more information, please refer to the [kymograph tutorial](https://lumicks-pylake.readthedocs.io/en/latest/tutorial/kymographs.html#correlating-with-channel-data).
 * Added support for loading two-color `TIF` files with [`ImageStack`](https://lumicks-pylake.readthedocs.io/en/latest/_api/lumicks.pylake.ImageStack.html#lumicks.pylake.ImageStack).
 * Made ([`CalibrationResults`](https://lumicks-pylake.readthedocs.io/en/latest/_api/lumicks.pylake.force_calibration.power_spectrum_calibration.CalibrationResults.html#calibrationresults)) callable to evaluate the fitted model power spectral density at the specified frequencies.
@@ -17,6 +18,10 @@
 * Warn when parameter estimates are hitting the fitting bounds when using [`lk.parameter_trace()`](https://lumicks-pylake.readthedocs.io/en/latest/_api/lumicks.pylake.parameter_trace.html).
 * Added `err_kappa` and `err_Rd` to force calibration results. These contain error estimates for the calibration constants propagated from the fitting errors.
 
+#### Other changes
+
+* Switch to notebook v7 with `ipympl`. See https://blog.jupyter.org/announcing-jupyter-notebook-7-8d6d66126dcf for more information. Note that this means that you have to have to invoke `%matplotlib widget` in notebooks where you previously used `%matplotlib notebook`.
+
 #### Bug fixes
 
 * Fixed statistical backing returning an incorrect value.

docs/examples/cas9_kymotracking/cas9_kymotracking.rst

@@ -25,12 +25,12 @@ required Python modules and choosing an interactive backend for `matplotlib`::
     import matplotlib.pyplot as plt
     import numpy as np
 
-    # Use notebook if you're in Jupyter Notebook
-    # %matplotlib notebook
-
-    # Use widget (depends on ipympl) if you're using Jupyter lab
+    # Use widget if you're using Jupyter lab or notebook
     %matplotlib widget
 
+    # Use notebook if you're in nbclassic
+    # %matplotlib notebook
+
 Download the kymograph data
 ---------------------------
 

docs/install.rst

@@ -1,7 +1,7 @@
 Installation
 ============
 
-Pylake can be installed on Windows, Linux and Mac and requires Python 3.9 or newer.
+Pylake can be installed on Windows, Linux and Mac and requires Python 3.10 or newer.
 
 .. note::
 
@@ -19,7 +19,7 @@ The easiest way to install Python and SciPy is with `Anaconda`_, a free scientif
 
 .. rubric:: Windows
 
-#. Go to the `Anaconda`_ website and download the Python 3.9 (or newer) installer.
+#. Go to the `Anaconda`_ website and download the Python 3.10 (or newer) installer.
 
 #. Run it and accept the default options during the installation.
 
@@ -29,13 +29,7 @@ The easiest way to install Python and SciPy is with `Anaconda`_, a free scientif
 
     .. code-block:: python
 
-        conda create -n pylake conda=23.7.2
-
-    .. note::
-
-        On Windows, there are issues with OpenSSL that preclude a successful installation on the latest version of conda.
-        This is why it is recommended to create an environment specifically with conda version `4.9` for now.
-        See `OpenSSL Error`_ for more information.
+        conda create -n pylake conda>=23.7.2
 
 #. The environment can then be activated::
 
@@ -69,7 +63,7 @@ This concludes the Pylake installation procedure. Check out the :doc:`Tutorial <
 
 .. rubric:: Linux
 
-#. Go to the `Anaconda`_ website and download the Python 3.9 (or newer) installer.
+#. Go to the `Anaconda`_ website and download the Python 3.10 (or newer) installer.
 
 #. Open a terminal window and run::
 
@@ -112,7 +106,7 @@ This concludes the Pylake installation procedure. Check out the :doc:`Tutorial <
 
 .. rubric:: macOS
 
-#. Go to the `Anaconda`_ website and download the Python 3.9 (or newer) installer.
+#. Go to the `Anaconda`_ website and download the Python 3.10 (or newer) installer.
 
 #. Run it and accept the default options during the installation.
 
@@ -147,7 +141,7 @@ This concludes the Pylake installation procedure. Check out the :doc:`Tutorial <
 Installation using pip
 ----------------------
 
-If you're already familiar with Python and have Python >= 3.9 installed, installing Pylake on Windows, Linux or Mac can be done using `pip`, Python's usual package manager::
+If you're already familiar with Python and have Python >= 3.10 installed, installing Pylake on Windows, Linux or Mac can be done using `pip`, Python's usual package manager::
 
     pip install lumicks.pylake
 
@@ -326,6 +320,33 @@ Important to note is that packages on `conda` and `pip` are typically *not* comp
 Frequently asked questions
 --------------------------
 
+**Why are the plots in my notebook not interactive?**
+
+Jupyter notebooks support the use of interactive figures.
+To enable interactive plots, you have to invoke the correct `magic commands <https://ipython.readthedocs.io/en/stable/interactive/magics.html>`_ in the notebook.
+When using Jupyter notebook, the following command will switch the `matplotlib` backend from the inline one (which renders images) to the interactive backend::
+
+    %matplotlib widget
+
+.. note::
+
+    Switching plotting backends typically requires you to restart the Jupyter kernel.
+
+**I tried using interactive plots using Jupyter notebook, but get the error `Javascript Error: IPython is not defined`**
+
+Starting from Pylake `1.4.0`, Pylake ships with a more recent version of Jupyter Notebook (`notebook >=7`).
+This newer version requires a different backend when using interactive figures: `%matplotlib notebook` has to be replaced by `%matplotlib widget`.
+Note that switching between interactive backends requires a kernel restart (select kernel -> restart from the menu at the top of the Jupyter Notebook).
+
+The reason for this change is that the backend which handled the interactive figures in `notebook<7` is no longer supported by Jupyter notebooks.
+Moving forward, `ipympl` will be used for interactive figures.
+See also the `matplotlib documentation <https://matplotlib.org/stable/users/explain/figure/interactive.html#jupyter-notebooks-jupyterlab>`_ on this.
+
+**I prefer the old interactive figures**
+
+It is still possible to use the old notebooks by installing `nbclassic` and starting the notebook with `jupyter nbclassic` instead of `jupyter notebook`.
+For more information on this, see the `nbclassic documentation <https://nbclassic.readthedocs.io/en/latest/nbclassic.html>`_.
+
 .. _OpenSSL Error:
 
 **I tried the installation instructions on Windows, but I get a CondaSSLError**
@@ -339,14 +360,14 @@ The full error message is::
 
     Exception: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/win-64/current_repodata.json (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))
 
-This issue has to be solved by conda. Until that happens, a possible solution is to use an older conda version::
+This issue has been solved upstream by conda. Make sure you install a new enough version::
 
-    conda create -n pylake conda=23.7.2
+    conda create -n pylake conda>=23.7.2
 
 And then follow the rest of the installation instructions.
-If you already have an environment named pylake, you can remove this environment, before creating it again with an older conda version. Another option is to create an environment with a different name, eg::
+If you already have an environment named pylake, you can remove this environment, before creating it again. Another option is to create an environment with a different name, eg::
 
-    conda create -n pylake2 conda=23.7.2
+    conda create -n pylake2 conda>=23.7.2
     conda activate pylake2
 
 **I tried the installation instructions, but I cannot import Pylake inside a Jupyter notebook**
@@ -412,30 +433,6 @@ If this happens, please try the following:
 * Restart the Jupyter Notebook and try again.
 
 
-**Why are the plots in my notebook not interactive?**
-
-To enable interactive plots, you have to invoke the correct `magic commands <https://ipython.readthedocs.io/en/stable/interactive/magics.html>`_
-in the notebook. When using Jupyter notebook, the following command will switch the `matplotlib` backend from the inline
-one (which renders images) to the interactive backend::
-
-    %matplotlib notebook
-
-You can also choose to install `ipympl`, which can perform better in some cases. You can install it with `pip`::
-
-    pip install ipympl
-
-or `conda`::
-
-    conda install -c conda-forge ipympl
-
-The `ipympl` backend can be activated by invoking the following magic command in a notebook::
-
-    %matplotlib widget
-
-*Note that switching backends typically requires you to restart the Jupyter kernel*. When using JupyterLab, `ipympl` is
-the only backend that provides interactive plots with Pylake.
-
-
 **Conda takes a long time to resolve the environment and then fails. What can I do?**
 
 Several packages depend on each other. Sometimes, finding a suitable collection of packages that is compatible can be

docs/tutorial/imagestack.rst

@@ -185,7 +185,7 @@ you can use the following function::
 
 .. image:: figures/imagestack/imagestack_correlated.png
 
-If the plot is interactive (for example, when `%matplotlib notebook` is used in a Jupyter notebook), you can click
+If the plot is interactive (for example, when `%matplotlib widget` is used in a Jupyter notebook), you can click
 on the left graph to select a particular force. The corresponding video frame will then automatically appear on the right.
 
 In some cases, additional processing may be needed, and we wish to have the data

docs/tutorial/nbwidgets.rst

@@ -8,12 +8,12 @@ Notebook Widgets
 When analyzing notebooks, it can be helpful to make use of interactive widgets. For this, we provide some widgets
 to help you analyze your data. To enable such widgets, start the notebook with::
 
-    # enable this line if you are using jupyter lab
-    %matplotlib widget
-
-    # enable this line if you are using jupyter notebook
+    # enable this line if you are using nbclassic
     # %matplotlib notebook
 
+    # enable this line if you are using jupyter notebook or lab
+    %matplotlib widget
+
 We can download the data needed for this tutorial directly from Zenodo using Pylake.
 Since we don't want it in our working folder, we'll put it in a folder called `"test_data"`::
 

docs/tutorial/scans.rst

@@ -183,4 +183,4 @@ The multi-frame confocal scans can also be correlated with a channel :class:`~lu
     multiframe_scan.plot_correlated(multiframe_file.force1x, adjustment=lk.ColorAdjustment([0, 0, 0], [4, 4, 4]))
     plt.show()
 
-Note that you need an interactive backend for this plot to work; instead of running ``%matplotlib inline`` at the top of the notebook, run ``%matplotlib notebook``. If some cells were already executed, you will need to restart the kernel as well.
+Note that you need an interactive backend for this plot to work; instead of running ``%matplotlib inline`` at the top of the notebook, run ``%matplotlib widget``. If some cells were already executed, you will need to restart the kernel as well.

lumicks/pylake/conftest.py

@@ -98,6 +98,17 @@ def configure_warnings():
         message=".*None into shape arguments as an alias for \\(\\) is.*",
     )
 
+    # This is a warning that gets issued by IPython when calling pytest from a notebook. The backend
+    # handling that used to be handled in IPython was moved to matplotlib. Until the mpl side of
+    # this is out a warning is issued. See:
+    #  - https://github.com/ipython/ipython/issues/14311
+    #  - https://github.com/ipython/ipython/pull/14371
+    warnings.filterwarnings(
+        "ignore",
+        category=DeprecationWarning,
+        message=".*backend2gui is deprecated since IPython.*",
+    )
+
 
 @pytest.fixture(autouse=True)
 def configure_mpl():

lumicks/pylake/nb_widgets/kymotracker_widgets.py

@@ -421,11 +421,10 @@ def _create_widgets(self):
         if not max([backend in plt.get_backend() for backend in ("nbAgg", "ipympl")]):
             raise RuntimeError(
                 (
-                    "Please enable an interactive matplotlib backend for this widget to work. In jupyter "
-                    "notebook you can do this by invoking either %matplotlib notebook or %matplotlib "
-                    "widget (the latter requires ipympl to be installed). In Jupyter Lab only the latter "
-                    "works. Please note that you may have to restart the notebook kernel for this to "
-                    "work."
+                    "Please enable an interactive matplotlib backend for this plot to work. In "
+                    "jupyter notebook or lab you can do this by invoking either "
+                    "%matplotlib widget. Please note that you may have to restart the notebook "
+                    "kernel for this to work."
                 )
             )
 

lumicks/pylake/nb_widgets/range_selector.py

@@ -273,11 +273,10 @@ def __init__(self, fd_curves):
         if not any(backend in plt.get_backend() for backend in ("nbAgg", "ipympl")):
             raise RuntimeError(
                 (
-                    "Please enable an interactive matplotlib backend for this plot to work. In jupyter"
-                    "notebook you can do this by invoking either %matplotlib notebook or %matplotlib "
-                    "widget (the latter requires ipympl to be installed). In Jupyter Lab only the latter "
-                    "works. Please note that you may have to restart the notebook kernel for this to "
-                    "work."
+                    "Please enable an interactive matplotlib backend for this plot to work. In "
+                    "jupyter notebook or lab you can do this by invoking either "
+                    "%matplotlib widget. Please note that you may have to restart the notebook "
+                    "kernel for this to work."
                 )
             )
 

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.black]
 line-length = 100
-target-version = ['py39']
+target-version = ['py310']
 exclude = '''
 (
   /(
@@ -20,6 +20,6 @@ exclude = '''
 [tool.isort]
 profile = "black"
 line_length = 100
-py_version=39
+py_version=310
 length_sort_sections = ["stdlib", "thirdparty", "firstparty", "localfolder"]
 lines_between_sections = 1