Classifypixelsunet doesn't load. Error while import keras

[ssitn]

I cannot make CellProfiler to load classifypixelsunet properly. It says 'No module named keras' while loading. I installed keras with anaconda promt. I even tried to uninstall and install it again. It still throws the same error upon start up of the CellProfiler. Here is the whole error message:

Could not load classifypixelsunet
Traceback (most recent call last):
File "CellProfiler\cellprofiler\modules_init_.py", line 325, in add_module
File "C:\Users\Sergey\AppData\Roaming\plugins\classifypixelsunet.py", line 67, in
import keras
ImportError: No module named keras

As I mentioned keras is installed:

Requirement already satisfied: keras in c:\users\User\anaconda3\lib\site-packages (2.3.1)

Am I doing something wrong?

[bethac07]

Hi,

Do you have CellProfiler installed from source or from downloading from the website? You can only use plugins like Classifypixelsunet when running from source.

[DeboraOlivier]

Hiya, I have a similar issue, so jumping on this thread. I am running my pipeline on a linux (ubuntu16) server and get this error message
Worker 9: File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 321, in placeholder Worker 9: x = tf.placeholder(dtype, shape=shape, name=name) Worker 9: AttributeError: 'module' object has no attribute 'placeholder'
newby in this so any help apprciated :-)
cheers

Debbi

[Scotchman0]

Hey Debbi,

what version of CellProfiler are you running? I Haven't tried running my baseline on a 16.04LTS version, but I've got a working version piped through 18.04 - might help to look through my code base and see if I did anything differently from your installation setup, especially regarding Keras installation and module re-names. (Theoretically it should work on 16.04 but it's better to go in clean - read the code first before you apply it).

https://github.com/Scotchman0/CellProfiler_Ubuntu_Installer

see also my previous issue on similar problem sets with classifypixelsunet and linux:
#90

I will say that I had a lot of trouble running ClassifyPixels until I abandoned python 2 and moved to python3.6 --> noting that the master branch of the cellprofiler repo excludes classifypixels unless you're running cellprofiler 3 or higher.

the "supported" linux instruction set will pull a cellprofiler2 version, and I had to essentially re-install all the plugins manually to get it to compile a cellprofiler3 version and integrate with classifypixels.

Hope this helps, good luck with your project.

~Will

[DeboraOlivier]

oh that is new, I thought classifyPixels was only working on Python2.7 (and not yet python 3). We had to create a specific environment for CellProfiler3.1.9 and install CP plugins in there too.
We used the following approach to install everything:

For 3.1.9 version

• clone https://github.com/CellProfiler/CellProfiler.git
• check out v3.1.9
• in setup.py, modify numpy==1.16.0, and scikit-image==0.14.2
  "boto3",
  "centrosome",
  "docutils",
  "h5py",
  "inflect",
  "javabridge",
  "joblib",
  "mahotas",
  "matplotlib>=2.0.0, !=2.1.0",
  "mysqlclient>=1.4.2",
  "numpy==1.16.0",
  "prokaryote==2.4.1",
  "python-bioformats==1.5.2",
  "pyzmq==15.3.0",
  "raven",
  "requests",
  "scikit-image==0.14.2",
  "scikit-learn",
  "scipy==1.1.0",
  "six
• create new conda env, python 2.7
• install with local user: pip install --user --editable .
• try to launch cellprofiler and see printed output
• install missing package with conda e.g. wx, ... see package list in: https://github.com/CellProfiler/CellProfiler/wiki/Conda-Installation
• missing wx ? : conda install -c daf wxpython=3.0.2.0

so in theory similar to you? doing this from a distance is not the easiest!

[DeboraOlivier]

@Scotchman0 which version of tensorflow did you have? might 2.0 be the issue?

[Scotchman0]

@DeboraOlivier Definitely possible that I was using the wrong version of Tensorflow. Sounds like we're doing more or less the same setup for installation - I don't think I adjusted any of the setup.py details with the exception of reducing minimum python from 3.7 to 3.6 for my installer - but that's because I failed to get it running with Cellprofiler2. ClassifyPixelsUnet is not technically part of the modules list on cellprofiler3 - but if you go in and add it manually it'll boot no problem when tensorflow/keras is in place properly.

I also didn't install using conda - did pip installations on my end, but my goal was 18.04LTS compatibility and it introduced some new issues.

Your approach looks good as far as I can tell - let's operate on the assumption you've installed it all appropriately and aren't (to our knowledge) missing anything.

Is there anything more to your error message? is this happening on app launch, or when you run the pipeline with a specific module? Can you run the app headless with a debug command baked in?

python CellProfiler.py --run --project /home//Desktop/testproject --log-level=DEBUG

See if it gives you any more insight into the problem?

(might be able to just call the cellprofiler applet and attach --log-level=DEBUG but sometimes it'll get finnicky if you try to do a headless launch without a project target.

then dump the output here in a .txt file attachment?

[bethac07]

We currently recommend protobuf==3.5.1, tensorflow==1.9.0, and keras==2.2.4 for that plugin.

…

On Mon, Mar 23, 2020 at 10:00 AM Will Russell ***@***.***> wrote: @DeboraOlivier <https://github.com/DeboraOlivier> Definitely possible that I was using the wrong version of Tensorflow. Sounds like we're doing more or less the same setup for installation - I don't think I adjusted any of the setup.py details with the exception of reducing minimum python from 3.7 to 3.6 for my installer - but that's because I failed to get it running with Cellprofiler2. ClassifyPixelsUnet is not technically part of the modules list on cellprofiler3 - but if you go in and add it manually it'll boot no problem when tensorflow/keras is in place properly. I also didn't install using cuda - did pip installations on my end, but my goal was 18.04LTS compatibility and it introduced some new issues. Your approach looks good as far as I can tell - let's operate on the assumption you've installed it all appropriately and aren't (to our knowledge) missing anything. Is there anything more to your error message? is this happening on app launch, or when you run the pipeline with a specific module? Can you run the app headless with a debug command baked in? python CellProfiler.py --run --project /home//Desktop/testproject --log-level=DEBUG See if it gives you any more insight into the problem? (might be able to just call the cellprofiler applet and attach --log-level=DEBUG but sometimes it'll get finnicky if you try to do a headless launch without a project target. then dump the output here in a .txt file attachment? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTI72YUDGD7ARNVKH2JLQTRI5TOZANCNFSM4J67KGNA> .

-- Beth Cimini, PhD Computational Biologist, Imaging Platform Broad Institute 415 Main St Room 5011 Cambridge, MA 02142 Current office number- (617) 714-8189 Pronouns - She/her/hers I will sometimes send or respond to emails outside of my local office hours, but I never expect responses outside of your local office hours.

[Scotchman0]

@DeboraOlivier Can we also get the output from:

pip freeze

[DeboraOlivier]

@Scotchman0 here it is

dolivierhub@ficoides:~$ pip freeze DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support absl-py==0.9.0 astor==0.8.1 attrs==15.2.0 backports.functools-lru-cache==1.6.1 backports.ssl-match-hostname==3.4.0.2 backports.weakref==1.0.post1 beautifulsoup4==4.4.1 boto3==1.12.21 botocore==1.15.21 cached-property==1.3.0 cachetools==3.1.1 cellh5==1.3.0 -e git+https://github.com/CellProfiler/CellProfiler.git@082930ea95add7b72243a4fa3d39ae5145995e9c#egg=CellProfiler centrosome==1.1.6 certifi==2019.11.28 chardet==3.0.4 click==7.1.1 cloudpickle==1.3.0 configobj==5.0.6 configparser==4.0.2 contextlib2==0.6.0.post1 cryptography==1.2.3 cycler==0.10.0 Cython==0.23.4 dask==1.2.2 decorator==4.0.6 deprecation==2.0.7 docker-compose==1.8.0 docker-py==1.9.0 dockerpty==0.4.1 docopt==0.6.2 docutils==0.15 enum34==1.1.10 funcsigs==1.0.2 functools32==3.2.3.post2 future==0.18.2 futures==3.3.0 gast==0.2.2 google-auth==1.11.3 google-auth-oauthlib==0.4.1 google-pasta==0.2.0 grpcio==1.27.2 gunicorn==19.4.5 gyp==0.1 h5py==2.10.0 hmmlearn==0.2.2 html5lib==0.999 idna==2.9 imagej==0.3.1 importlib-metadata==1.5.0 inflect==3.0.2 ipaddr==2.1.11 ipaddress==1.0.16 ipywidgets==6.0.0 javabridge==1.0.18 jmespath==0.9.5 joblib==0.14.1 jsonschema==2.5.1 Keras==1.2.2 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 kiwisolver==1.1.0 Landscape-Client==16.3+bzr834 libvirt-python==1.3.1 lxml==3.5.0 mahotas==1.4.9 Markdown==3.1.1 matplotlib==2.2.5 microscopeimagequality==0.1.0.dev5 mock==3.0.5 mysqlclient==1.4.6 ndg-httpsclient==0.4.0 networkx==1.11 nose==1.3.7 numpy==1.16.0 oauthlib==3.1.0 olefile==0.46 opt-einsum==2.3.2 packaging==20.3 PAM==0.4.2 pandas==0.24.2 pathlib2==2.3.5 pbr==3.1.1 Pillow==4.3.0 prokaryote==2.4.1 protobuf==3.11.3 py==1.4.31 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycrypto==2.6.1 pygobject==3.20.0 pyOpenSSL==0.15.1 pyparsing==2.0.3 pyserial==3.0.1 Pyste==0.9.10 pytest==2.8.7 python-apt==1.1.0b1+ubuntu0.16.4.8 python-bioformats==1.5.2 python-dateutil==2.8.1 python-debian==0.1.27 pytz==2014.10 PyWavelets==1.0.3 PyYAML==3.12 pyzmq==15.3.0 raven==6.10.0 requests==2.23.0 requests-oauthlib==1.3.0 rsa==4.0 s3transfer==0.3.3 scandir==1.10.0 scikit-image==0.14.2 scikit-learn==0.20.4 scipy==1.2.2 scour==0.32 seaborn==0.7.1 service-identity==16.0.0 six==1.14.0 stevedore==1.26.0 subprocess32==3.5.4 tensorboard==2.1.0 tensorflow==2.1.0 tensorflow-estimator==2.1.0 termcolor==1.1.0 texttable==0.8.1 Theano==0.8.2 toolz==0.8.2 Twisted==16.0.0 urllib3==1.25.8 virtualenv==15.1.0 virtualenv-clone==0.2.6 virtualenvwrapper==4.8.1 websocket-client==0.18.0 Werkzeug==1.0.0 wndcharm==0.9.14+dc3dc7clocalmod wrapt==1.12.1 wxPython==3.0.2.0 wxPython-common==3.0.2.0 zeroc-ice==3.6.3 zipp==1.2.0 zope.interface==4.1.3

[DeboraOlivier]

@bethac07 that's for the ubuntu, right? will it work if we create a virtual environment for cellprofiler install? just so as not to screw up the rest of the server for others?

[Scotchman0]

protobuf==3.11.3,
tensorflow==2.1.0
Keras==1.2.2

Are all different than the recommended ones @bethac07 mentioned before - might be worth trying to resolve with an update to keras, since protobuf and tensorflow are all on later threads.

Re Ubuntu - containerizing this might not be a bad idea - especially if you're using this production server for other processes - (or at the very minimum being able to restore from snapshots if it's virtually hosted).

[bethac07]

That's based on how we got this working on your computer in the past. In general, a virtual environment is a good idea; these are the package version things I recommended to you when we were getting this working on your laptop, in general they should be the same for any machine. [image: image.png]

…

On Mon, Mar 23, 2020 at 10:28 AM DeboraOlivier ***@***.***> wrote: @bethac07 <https://github.com/bethac07> that's for the ubuntu, right? will it work if we create a virtual environment for cellprofiler install? just so as not to screw up the rest of the server for others? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTI725ZO24EOHWIMNJR5RDRI5WYLANCNFSM4J67KGNA> .

-- Beth Cimini, PhD Computational Biologist, Imaging Platform Broad Institute 415 Main St Room 5011 Cambridge, MA 02142 Current office number- (617) 714-8189 Pronouns - She/her/hers I will sometimes send or respond to emails outside of my local office hours, but I never expect responses outside of your local office hours.

[DeboraOlivier]

well - bad note keeping from my side (though now all this conversation has been kept for future generations in a drive document - jumping between PC and Mac and now linux doesn't help with book-keeping.
We will create and environment and install with the versions as specified (and hope for the best).
That would be with python2.7 still (think we used 2.7 on my mac)
sorry I drop on/off due to children needing my attention.... confinment!!

[DeboraOlivier]

ok, virtual environment in python 2.7, protobuf==3.5.1, tensorflow==1.9.0, and keras==2.2.4
IT WORKS!! (though I still have a few errors upon startup - see below) but it is running.
Now the next step is to link to the data server...
thanks for the help guys!
I put below my pip freeze for future reference and the errors I get.

Errors
cellprofiler:26927): Gdk-WARNING **: gdk_window_set_icon_list: icons too large Traceback (most recent call last): File "/home/dolivierhub/CellProfiler/cellprofiler/gui/moduleview.py", line 4180, in GetAttr attr.IncRef() # OH so bogus, don't refcount = bus error File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_core.py", line 852, in IncRef return _core_.RefCounter_IncRef(*args, **kwargs) wx._core.PyAssertionError: C++ assertion "Assert failure" failed at ../src/gtk/window.cpp(4096) in GTKSendPaintEvents(): unsupported background style Traceback (most recent call last): File "/home/dolivierhub/CellProfiler/cellprofiler/gui/moduleview.py", line 4180, in GetAttr attr.IncRef() # OH so bogus, don't refcount = bus error File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_core.py", line 852, in IncRef return _core_.RefCounter_IncRef(*args, **kwargs) wx._core.PyAssertionError: C++ assertion "Assert failure" failed at ../src/gtk/window.cpp(4096) in GTKSendPaintEvents(): unsupported background style Traceback (most recent call last): File "/home/dolivierhub/CellProfiler/cellprofiler/gui/moduleview.py", line 4180, in GetAttr attr.IncRef() # OH so bogus, don't refcount = bus error File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_core.py", line 852, in IncRef return _core_.RefCounter_IncRef(*args, **kwargs) wx._core.PyAssertionError: C++ assertion "Assert failure" failed at ../src/gtk/window.cpp(4096) in GTKSendPaintEvents(): unsupported background style

PIP-FREEZE
absl-py==0.9.0 astor==0.8.1 attrs==15.2.0 backports.functools-lru-cache==1.6.1 backports.ssl-match-hostname==3.4.0.2 backports.weakref==1.0.post1 beautifulsoup4==4.4.1 boto3==1.12.21 botocore==1.15.21 cached-property==1.3.0 cachetools==3.1.1 cellh5==1.3.0 -e git+https://github.com/CellProfiler/CellProfiler.git@082930ea95add7b72243a4fa3d39ae5145995e9c#egg=CellProfiler centrosome==1.1.6 certifi==2019.11.28 chardet==3.0.4 click==7.1.1 cloudpickle==1.3.0 configobj==5.0.6 configparser==4.0.2 contextlib2==0.6.0.post1 cryptography==1.2.3 cycler==0.10.0 Cython==0.23.4 dask==1.2.2 decorator==4.0.6 deprecation==2.0.7 docker-compose==1.8.0 docker-py==1.9.0 dockerpty==0.4.1 docopt==0.6.2 docutils==0.15 enum34==1.1.10 funcsigs==1.0.2 functools32==3.2.3.post2 future==0.18.2 futures==3.3.0 gast==0.2.2 google-auth==1.11.3 google-auth-oauthlib==0.4.1 google-pasta==0.2.0 grpcio==1.27.2 gunicorn==19.4.5 gyp==0.1 h5py==2.10.0 hmmlearn==0.2.2 html5lib==0.999 idna==2.9 imagej==0.3.1 importlib-metadata==1.5.0 inflect==3.0.2 ipaddr==2.1.11 ipaddress==1.0.16 ipywidgets==6.0.0 javabridge==1.0.18 jmespath==0.9.5 joblib==0.14.1 jsonschema==2.5.1 Keras==2.2.4 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 kiwisolver==1.1.0 Landscape-Client==16.3+bzr834 libvirt-python==1.3.1 lxml==3.5.0 mahotas==1.4.9 Markdown==3.1.1 matplotlib==2.2.5 microscopeimagequality==0.1.0.dev5 mock==3.0.5 mysqlclient==1.4.6 ndg-httpsclient==0.4.0 networkx==1.11 nose==1.3.7 numpy==1.16.0 oauthlib==3.1.0 olefile==0.46 opt-einsum==2.3.2 packaging==20.3 PAM==0.4.2 pandas==0.24.2 pathlib2==2.3.5 pbr==3.1.1 Pillow==4.3.0 prokaryote==2.4.1 protobuf==3.5.1 py==1.4.31 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycrypto==2.6.1 pygobject==3.20.0 pyOpenSSL==0.15.1 pyparsing==2.0.3 pyserial==3.0.1 Pyste==0.9.10 pytest==2.8.7 python-apt==1.1.0b1+ubuntu0.16.4.8 python-bioformats==1.5.2 python-dateutil==2.8.1 python-debian==0.1.27 pytz==2014.10 PyWavelets==1.0.3 PyYAML==3.12 pyzmq==15.3.0 raven==6.10.0 requests==2.23.0 requests-oauthlib==1.3.0 rsa==4.0 s3transfer==0.3.3 scandir==1.10.0 scikit-image==0.14.2 scikit-learn==0.20.4 scipy==1.2.2 scour==0.32 seaborn==0.7.1 service-identity==16.0.0 six==1.14.0 stevedore==1.26.0 subprocess32==3.5.4 tensorboard==1.9.0 tensorflow==1.9.0 tensorflow-estimator==1.13.0 termcolor==1.1.0 texttable==0.8.1 Theano==0.8.2 toolz==0.8.2 Twisted==16.0.0 urllib3==1.25.8 virtualenv==15.1.0 virtualenv-clone==0.2.6 virtualenvwrapper==4.8.1 websocket-client==0.18.0 Werkzeug==1.0.0 wndcharm==0.9.14+dc3dc7clocalmod wrapt==1.12.1 wxPython==3.0.2.0 wxPython-common==3.0.2.0 zeroc-ice==3.6.3 zipp==1.2.0 zope.interface==4.1.3

[ErinWeisbart]

I am closing this issue because it refers to a plugin that we are not currently maintaining.