Management of python package conflicts between extensions

[aTamaz]

Is your feature request related to a problem? Please describe.

I have a Slicer extension that integrates a deep learning method with a Python interface into Slicer by 1. make sure required python packages are installed, and 2. launching the computationally-intensive task in a separate Python process.

Currently, all Slicer extensions install python packages in the same environment -- the one that is also shared with the Slicer core. Therefore, extension developers have to

1. manage version conflicts between their package requirements and packages used by Slicer (e.g., pillow, SimpleITK)
2. manage conflicts with packages installed by other extensions (by uninstalling these packages and installing the required version).

A good example of the emerging complexity can be found here.

For the extension user, it means that packages are potentially being re-installed every time they switch between extensions.

Describe the solution you'd like

Ideally, an extension developer would have a possibility to specify their package requirements in Python, which are then installed in an isolated environment, allowing for co-existence of multiple versions of the same package within a Slicer installation (similar to a python virtual environment). pipx may be useful in this context.

Also see Issue #7707 .

Describe alternatives you've considered

Current solution is to uninstall conflicting packages and install the required version of the package.

Additional context

--

[fedorov]

Current solution is to uninstall conflicting packages and install the required version of the package.

And then probably restart Slicer, and inform the user that Slicer needs to be restarted ... we seem to experience similar issue in ImagingDataCommons/SlicerIDCBrowser#41.

[pieper]

As we discussed at project week, it really seems that just creating and running a venv for the CLI would handle this case well.

Or maybe we could use the --user option and PYTHONUSERBASE variable to point pip to a per-module location and then prefix that to PYTHONPATH before running the cli.