Java GC overhead limit exceeded CP 3.0.0 Ubuntu 16.04

[anou85]

Hi,
I've finally managed to install the CP version 3.0.0 (from the github master branch, sudo -H pip install --editable .) on my Ubuntu 16.04. It installed fine, and it starts up, but as soon as I try to load metadata, it kind of stops. A pop-up error doesn't come up, but the terminal indicates a java GC overhead limit exceeded:

Traceback (most recent call last):
  File "/home/anja/CellProfiler-master/cellprofiler/gui/moduleview.py", line 1267, in callback
    self.__on_do_something(event, setting)
  File "/home/anja/CellProfiler-master/cellprofiler/gui/moduleview.py", line 2231, in __on_do_something
    setting.on_event_fired()
  File "/home/anja/CellProfiler-master/cellprofiler/setting.py", line 2140, in on_event_fired
    self.__callback(*self.__args)
  File "/home/anja/CellProfiler-master/cellprofiler/modules/metadata.py", line 663, in <lambda>
    lambda: self.do_update_metadata(group),
  File "/home/anja/CellProfiler-master/cellprofiler/modules/metadata.py", line 1044, in do_update_metadata
    metadata = get_omexml_metadata(url=url)
  File "/usr/local/lib/python2.7/dist-packages/bioformats/formatreader.py", line 1025, in get_omexml_metadata
    xml = jutil.run_script(script, dict(path=rdr.path, reader = rdr.rdr))
  File "/usr/local/lib/python2.7/dist-packages/javabridge/jutil.py", line 384, in run_script
    scope, script, "<java-python-bridge>", 0, None)
  File "/usr/local/lib/python2.7/dist-packages/javabridge/jutil.py", line 887, in call
    result = fn(*nice_args)
  File "/usr/local/lib/python2.7/dist-packages/javabridge/jutil.py", line 854, in fn
    raise JavaException(x)
javabridge.jutil.JavaException: GC overhead limit exceeded

this is my pip freeze:

adium-theme-ubuntu==0.3.4
altgraph==0.15
attrs==17.4.0
backports.functools-lru-cache==1.5
beautifulsoup4==4.4.1
cellh5==1.3.0
CellProfiler==3.0.0
centrosome==1.0.9
certifi==2018.1.18
chardet==3.0.4
contextlib2==0.5.5
csvfilter==0.3.1
cycler==0.10.0
Cython==0.24.1
decorator==4.2.1
defer==1.0.6
dis3==0.1.2
docutils==0.14
fluendo-eula==0.6
funcsigs==1.0.2
future==0.16.0
gWakeOnLan==0.5.1
h5py==2.7.1
hmmlearn==0.2.0
html5lib==0.999
idna==2.6
inflect==0.2.5
javabridge==1.0.15
joblib==0.11
libtiff==0.4.2
lxml==3.5.0
macholib==1.9
mahotas==1.4.4
matplotlib==2.1.2
MySQL-python==1.2.5
mysqlclient==1.3.7
networkx==2.1
numpy==1.11.3
pandas==0.22.0
pefile==2017.11.5
Pillow==3.3.1
pkginfo==1.4.1
pluggy==0.6.0
prokaryote==2.3.2
py==1.5.2
pycrypto==2.6.1
pygobject==3.20.0
pygpgme==0.3
PyInstaller==3.3.1
pyparsing==2.0.3
pytest==3.4.0
python-apt==1.1.0b1
python-bioformats==1.3.2
python-dateutil==2.4.2
python-debian==0.1.27
python-distutils-extra==2.39
pytz==2014.10
PyWavelets==0.5.2
pyzmq==15.3.0
raven==6.5.0
requests==2.18.4
requests-toolbelt==0.8.0
scikit-image==0.13.1
scikit-learn==0.19.1
scipy==0.17.0
scour==0.32
six==1.10.0
subprocess32==3.2.7
tqdm==4.19.5
twine==1.9.1
unity-lens-photos==1.0
urllib3==1.22
wxPython==3.0.2.0
wxPython-common==3.0.2.0

I've already increased the java memory preferences in CP, doesn't change anything. Do I need different versions of javabridge/ bioformats?

Cheers,
Anja

[AetherUnbound]

Is there anyway you can package or upload the image you're trying to load? I'd like to try and test it on my machine.

[anou85]

@AetherUnbound, thanks for getting back to me. It's no problem loading the same images in the 2.2.0 pipeline on an older computer, but will upload them for you. 96 well plate with 2 channels and 6 photos per well -> 576 images. But I've loaded even more with the 2.2.0 version
https://drive.google.com/open?id=1Y-aggRO2csIvVtW0odvSBOVzDUNUS6ev

[mcquin]

You could simply be running out of heap space. You can try to increase the amount of memory allocated to the JVM in the CellProfiler preferences, or by setting the JAVA_OPTS environment variable:

set JAVA_OPTS="-Xms128m -Xmx256m"

More details

[anou85]

I mentioned above that I've already tried to increased the java memory preference (multiple GiB I think). Plus the CP 2.2.0 runs with 512m just fine

[mcquin]

I apologize, I missed that point in your previous description. We'll continue to investigate with the data you've provided. Thank you!

[anou85]

I appreciate your help :)
Does it make sense to try with a newer release of the master branch...?

[mcquin]

I don't believe we've committed any changes at this point which would resolve the issue. We'll let you know when we have another update.

[anou85]

Thanks a lot

[bethac07]

Related to #1938