script py3 fixes #354

will-moore · 2019-12-03T15:07:33Z

Fixes adding Shapes to panels when exporting as TIFFs.
To test:

Add a bunch of ROIs to some image panels
Export as TIFF

will-moore · 2019-12-04T09:30:07Z

Travis tests are failing with ModuleNotFoundError: No module named 'numpy' so we need numpy for tests. I was looking to see where this is handled in omero-mapr, which has the same requirement, but I don't see it?

joshmoore · 2019-12-04T09:32:21Z

Add it to setup.py? Pretty sure I rolled back my commit since you said it wasn't necessary. (ome/omero-mapr@328c51e)

will-moore · 2019-12-04T10:24:38Z

Re: ome/omero-mapr@328c51e That was due to a 'warning' message from webgateway since it uses numpy for some things, but they weren't being used for mapr.
However, as discussed - since BlitzGateway uses numpy for some things, we should add it as a requirement of omero-py. See ome/omero-py#139

snoopycrimecop · 2019-12-06T00:28:27Z

Conflicting PR. Removed from build OMERO-plugins-push#108. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

will-moore · 2019-12-06T11:30:48Z

Tried exploring omero-test-infra...

$ NOCLEAN=true .omero/docker app
...
running build_ext
Traceback (most recent call last):
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1028, in process_custom_settings
    setattr(module, global_name, mapping(global_value))
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)


$ .omero/compose exec web bash
bash-4.2$ whoami
omero-web

./omero config get
cd ../lib
./python3
>>> import os
>>> os.sys.path
['', '/opt/omero/web/OMERO.web/lib/python', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/opt/omero/web/venv3/lib64/python3.6/site-packages', '/opt/omero/web/venv3/lib/python3.6/site-packages']

pwd
/opt/omero/web/venv3/lib64
bash-4.2$ ../bin/python3
>>> import omero
>>> from omeroweb import settings 
Traceback (most recent call last):
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1028, in process_custom_settings
    setattr(module, global_name, mapping(global_value))
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1043, in <module>
    'DEPRECATED_SETTINGS_MAPPINGS')
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1032, in process_custom_settings
    (global_name, key, global_value, e.message, description))
AttributeError: 'JSONDecodeError' object has no attribute 'message'

BUT, can't edit python code (not root)....

$ .omero/compose exec -u root web bash
[root@22fb5e33fb9d setup]# whoami
root
# ps -aux | grep django
..../opt/omero/web/venv3/bin/python3  .... /opt/omero/web/OMERO.web/
# cd /opt/omero/web/venv3/bin
# python3
>>> import omero .  # ModuleNotFound

# cd /opt/omero/web/venv3/
# bin/python3

vi /opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py

Found with print(global_value) in omeroweb/settings.py process_custom_settings() that omero.web.apps value [omero_figure] was not quoted correctly (should be ["omero_figure"]

snoopycrimecop · 2019-12-07T00:28:49Z

Conflicting PR. Removed from build OMERO-plugins-push#109. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

snoopycrimecop · 2019-12-08T00:29:00Z

Conflicting PR. Removed from build OMERO-plugins-push#110. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- setup.py

--conflicts

will-moore · 2019-12-10T14:30:57Z

Figure export tests are failing on travis due to missing reportlab.
Locally I find it is available on the server container in py2 but not Python-3:

# remove any old docker containers
$ docker rm -f $(docker ps -aq)
$ NOCLEAN=true .omero/docker app --verbose
# shell into server container...
$ .omero/compose exec omero bash

bash-4.2$ cd /opt/omero/server/OMERO.server/
bash-4.2$ ls
LICENSE.txt  bin  etc  history.rst  lib  omero.envvars	share  sql  var
bash-4.2$ python
Python 2.7.5 (default, Aug  4 2017, 00:39:18) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import reportlab

# find where omero is running - python3
$ ps -aux | grep omero
omero-s+   141  0.1  0.0 231300    72 ?        S    14:17   0:01 /opt/omero/server/venv3/bin/python /opt/omero/server/OMERO.server/bin/omero admin start --foreground

bash-4.2$ /opt/omero/server/venv3/bin/python
Python 3.6.8 (default, Aug  7 2019, 17:28:10) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import reportlab 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'reportlab'

Reportlab is installed in ./.omeroci/app-srv with

yum -y install python-reportlab python-markdown

Need to ensure it is installed for python3...

will-moore · 2019-12-10T14:34:25Z

Looks like /opt/omero/server/venv3/bin/pip install reportlab should work...

jburel · 2019-12-10T14:43:52Z

it is probably python3-reportlab

jburel · 2019-12-10T14:45:12Z

now that everything is installed in virtualenv
the steps for figure should be split into 2:

install figure in web venv
install script in server venv with its dependencies

will-moore · 2019-12-10T14:49:27Z

Locally I'm seeing an issue with install of scripts:

>       assert script_service.getParams(id) is not None

test/integration/test_figure_scripts.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = 13cc1d5d-9208-47ea-ba37-e1acfc79c78c/e59c8e23-35ec-4d74-8d42-d8d629844fd7omero.api.IScript -t -e 1.1:tcp -h 172.22.0.4 -p 36937 -t 60000
scriptID = 63, _ctx = None

    def getParams(self, scriptID, _ctx=None):
>       return _M_omero.api.IScript._op_getParams.invoke(self, ((scriptID, ), _ctx))
E       omero.InternalException: exception ::omero::InternalException
E       {
E           serverStackTrace = 
E           serverExceptionClass = 
E           message = Sha1s don't match! expected 4cfaf0e9cbf68fb5dc25bc2816f8c3b83a7248e7, found cd16eafce5be6f5f47bf66ce78d6ac730115ce88
E       }

/opt/omero/web/venv3/lib64/python3.6/site-packages/omero_api_IScript_ice.py:842: InternalException
====================== 5 failed, 1 passed in 6.85 seconds ======================

snoopycrimecop · 2019-12-10T23:00:41Z

Conflicting PR. Removed from build OMERO-plugins-push#142. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- .omeroci/app-srv
- setup.py

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#113. See the console output for more details.

snoopycrimecop · 2019-12-11T00:43:48Z

Conflicting PR. Removed from build OMERO-plugins-push#127. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- setup.py
- .omeroci/app-srv

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#114. See the console output for more details.

snoopycrimecop · 2019-12-12T00:44:52Z

Conflicting PR. Removed from build OMERO-plugins-push#128. See the console output for more details.
Possible conflicts:

PR Fix page count #351 will-moore 'Handle unicode figure names in export'
- .omeroci/app-srv
- setup.py

~~--conflicts~~ Conflict resolved in build OMERO-plugins-push#144. See the console output for more details.

jburel · 2019-12-12T11:31:34Z

Tested the functionality of this PR
Figure exported as TIFF contains the various shapes and the settings are preserved.
@will-moore will first need to sort out this PR before we can merge

will-moore · 2019-12-12T13:21:38Z

@jburel Rebased to have only my 1-character fix!

jburel · 2019-12-13T09:29:59Z

Retested today. It works as expected
Merging

will-moore mentioned this pull request Dec 4, 2019

Add numpy to requirements in setup.py ome/omero-py#139

Merged

will-moore force-pushed the script_py3_fixes branch from a537ba6 to 0b14010 Compare December 5, 2019 16:36

will-moore mentioned this pull request Dec 6, 2019

Fix app-config omero.web.apps formatting ome/omero-test-infra#53

Merged

will-moore changed the title ~~Fix adding Shapes to panels in TIFF export~~ script py3 fixes Dec 10, 2019

snoopycrimecop mentioned this pull request Dec 11, 2019

flake8 fix #356

Merged

Fix adding Shapes to panels in TIFF export

d669ff3

will-moore force-pushed the script_py3_fixes branch from c248a6c to d669ff3 Compare December 12, 2019 11:45

jburel merged commit cc55693 into ome:master Dec 13, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

script py3 fixes #354

script py3 fixes #354

will-moore commented Dec 3, 2019 •

edited

Loading

will-moore commented Dec 4, 2019

joshmoore commented Dec 4, 2019

will-moore commented Dec 4, 2019

snoopycrimecop commented Dec 6, 2019

will-moore commented Dec 6, 2019 •

edited

Loading

snoopycrimecop commented Dec 7, 2019

snoopycrimecop commented Dec 8, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

jburel commented Dec 10, 2019

jburel commented Dec 10, 2019

will-moore commented Dec 10, 2019

snoopycrimecop commented Dec 10, 2019 •

edited

Loading

snoopycrimecop commented Dec 11, 2019 •

edited

Loading

snoopycrimecop commented Dec 12, 2019 •

edited

Loading

jburel commented Dec 12, 2019

will-moore commented Dec 12, 2019

jburel commented Dec 13, 2019

script py3 fixes #354

script py3 fixes #354

Conversation

will-moore commented Dec 3, 2019 • edited Loading

will-moore commented Dec 4, 2019

joshmoore commented Dec 4, 2019

will-moore commented Dec 4, 2019

snoopycrimecop commented Dec 6, 2019

will-moore commented Dec 6, 2019 • edited Loading

snoopycrimecop commented Dec 7, 2019

snoopycrimecop commented Dec 8, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

jburel commented Dec 10, 2019

jburel commented Dec 10, 2019

will-moore commented Dec 10, 2019

snoopycrimecop commented Dec 10, 2019 • edited Loading

snoopycrimecop commented Dec 11, 2019 • edited Loading

snoopycrimecop commented Dec 12, 2019 • edited Loading

jburel commented Dec 12, 2019

will-moore commented Dec 12, 2019

jburel commented Dec 13, 2019

will-moore commented Dec 3, 2019 •

edited

Loading

will-moore commented Dec 6, 2019 •

edited

Loading

snoopycrimecop commented Dec 10, 2019 •

edited

Loading

snoopycrimecop commented Dec 11, 2019 •

edited

Loading

snoopycrimecop commented Dec 12, 2019 •

edited

Loading