Making CellProfiler plugins easier to install

[callum-jpg]

@fefossa @ErinWeisbart

How should a user install CellProfiler plugins?

1. Have CellProfiler be installed when pip installing plugin dependencies
   1. Could therefore pin cellprofiler version to be used with plugins (eg 4.2.5)
2. Assume that the user already has CellProfiler installed when installing plugin dependencies
   1. This method makes it harder to account for versions of CellProfiler that have not been tested with the plugins

For all options, how do we handle installing of these dependencies? Creating environments that support CellPose (PyTorch) and Stardist (Tensorflow) has previously been shown to be possible, but it notoriously difficult, so these should remain split for now. However, for a lot of other non-deep learning plugins the dependencies are much simpler.

The below solution assumes setup.py is being used, but other package dependency management methods like poetry could be used.

With this in mind, we could use extras_require in setup.py. This would mean a user can install just the basic plugins if they would like, or either cellpose+basic plugins or stardist+basic plugins:

install_deps = [
            cellprofiler=4.2.5
            cellprofiler-core=4.2.5
                ]

basic_deps = [
            **non-deep learning dependencies**
]

cellpose_deps = [
            pytorch
]

stardist_deps = [
            tensorflow
]

setup(
    ...
    install_requires = install_deps
    extras_require = {
      'basic': basic_deps,
      'cellpose': basic_deps + cellpose_deps,
      'stardist': basic_deps + stardist_deps,
    },
    ...
)

How would installing using the above package look? Following git cloning of the CellProfiler-plugins repo and cd'ing inside using your terminal, you could run pip install -e .[basic] or pip install -e .[cellpose].

It's also possible to programmatically set the plugin directory with:

import cellprofiler_core.preferences
cellprofiler_core.preferences.set_plugin_directory("example_plugin_directory")

This would be relevant if install option 1 above was considered. Otherwise this would be manually set by the user in the GUI. That said, this will be required for creating a testing github action for a plugins pipeline.

The dependencies for all of the plugins that are in the root directory of the repo are as follows (this was acquired using the python package pipreq):

cellpose==2.2 # Deep learning
cellpose_omni==0.9.1 # Deep learning
cellprofiler==4.2.5
cellprofiler_core==4.2.5
centrosome==1.2.1
csbdeep==0.7.3 # Deep learning
matplotlib==3.7.1
ncolor==1.2.1
numpy==1.24.3
omnipose==0.4.4 # Deep learning
pandas==2.0.1
scipy==1.10.1
skimage
stardist==0.8.3 # Deep learning
tensorflow==2.12.0 # Deep learning
torch==2.0.0 # Deep learning

The non-deep learning dependencies should be easy. Even the pandas dependency could be removed by refactoring variancetransform.py to ensure the as much overlap with CellProfiler's own dependencies.

[bethac07]

Tagging in @hinerm here too. Thanks for getting this kicked off!

[bethac07]

This would be relevant if install option 1 above was considered. Otherwise this would be manually set by the user in the GUI. That said, this will be required for creating a testing github action for a plugins pipeline.

I don't think you'll need to do this, thankfully! I think just calling CellProfiler headless with the --plugins-directory flag gets you what you need - see example in DCP worker file.

[bethac07]

install_deps = [
            cellprofiler=4.2.5
            cellprofiler-core=4.2.5
                ]

I think we want to avoid setting an exact CellProfiler version here; maybe an ~= version?

[bethac07]

cellpose==2.2
cellpose_omni==0.9.1

These need DL tags too!

[bethac07]

Even the pandas dependency could be removed by refactoring variancetransform.py to ensure the as much overlap with CellProfiler's own dependencies.

Yup, I think we definitely want to do this, the grad student using it said he had actually already done it and it was working for him. I could reach out to ping him to ask for a PR maybe?

[callum-jpg]

That would be perfect

[bethac07]

Email sent

[hinerm]

The dependencies for all of the plugins that are in the root directory of the repo are as follows (this was acquired using the python package pipreq):

Just noting that it looks like the runimagej-script.py requirements were missed by pipreq

[callum-jpg]

@hinerm By any chance do you have an example of installing imagej in a headless environment? The goal is to install ImageJ in ubuntu through github actions and then run cellprofiler in headless mode with a pipeline that contains runimagejscript, so any help with getting this up and running would be immensely appreciated.

[hinerm]

@callum-jpg I think the best way to use ImageJ headless is via pyimagej. That's what we're already doing in runimagejscript. You do not need to install ImageJ explicitly - it can be downloaded automatically via pyimagej and cached (by default in the ~/.jgo directory). If you need to show runimagejscript with an existing ImageJ installation, you can just get it from the downloads page and point runimagejscript at it.