Reactivate infra #57

will-moore · 2019-11-28T13:33:04Z

This tests omero-test-infra when the omero_mapr/__init__.py imports the urls.py.
I think we often don't see the failure of urls.py to import because it doesn't happen when the omero_mapr module itself is imported.
This is on top of #56 (alternative sibling PR since we are both working on this)

will-moore · 2019-11-28T15:51:16Z

After that last commit I see:

+ python .//setup.py test
testimport urls.py...
ImportError
running pytest

but no stacktrace to show how the import failed.

However, I see this later:

Running PyYAML-5.2b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-8zzdcwu0/PyYAML-5.2b1/egg-dist-tmp-rvh9xhou
Traceback (most recent call last):
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 33, in <module>
    from omero import ApiUsageException, ServerError
ImportError: cannot import name 'ApiUsageException'

However, running .omero/docker app locally I instead see the same import error at a different point

+ python setup.py clean
testimport urls.py...
ImportError
running clean
Traceback (most recent call last):
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 35, in <module>
    from omero import ApiUsageException, ServerError
ImportError: cannot import name 'ApiUsageException'

will-moore · 2019-11-28T15:53:21Z

Locally, this fails:

>>> from omero import ApiUsageException
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name ApiUsageException

but works if you first do import omero.clients (which fails flake8)!

After adding import omero.clients in the last commit above, we now fail travis with

+ python setup.py clean
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 38, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 38, in <module>
    from .mapr_settings import mapr_settings
  File "/omero-mapr/omero_mapr/mapr_settings.py", line 73, in <module>
    TEMPLATES = getattr(settings, 'TEMPLATES', [])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

will-moore · 2019-11-28T16:41:46Z

After removing the usage of ApiException and ServerError, we are still seeing

+ python setup.py clean
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 38, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 23, in <module>
    from . import views
  File "/omero-mapr/omero_mapr/views.py", line 35, in <module>
    from .mapr_settings import mapr_settings
  File "/omero-mapr/omero_mapr/mapr_settings.py", line 73, in <module>
    TEMPLATES = getattr(settings, 'TEMPLATES', [])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

will-moore · 2019-11-28T23:22:30Z

Failing with:

+ python setup.py clean
ERROR:omeroweb.webgateway.views:No numpy installed
testimport urls.py...
Traceback (most recent call last):
  File "setup.py", line 41, in <module>
    version = __import__('omero_mapr').get_version()
  File "/omero-mapr/omero_mapr/__init__.py", line 30, in <module>
    from . import urls
  File "/omero-mapr/omero_mapr/urls.py", line 36, in <module>
    DEFAULT_CONFIG = list(mapr_settings.CONFIG.keys())[0]
IndexError: list index out of range

snoopycrimecop · 2019-11-29T00:28:29Z

Conflicting PR. Removed from build OMERO-plugins-push#101. See the console output for more details.
Possible conflicts:

PR Reactivate test-infra with 5.6.0-m2 #56 joshmoore 'Reactivate test-infra with 5.6.0-m2'
- omero_mapr/views.py

--conflicts

will-moore · 2019-11-29T09:50:32Z

To test locally need to

docker pull openmicroscopy/omero-web:5.6

will-moore · 2019-11-29T17:43:17Z

NOCLEAN=true .omero/docker app
.omero/compose exec web bash
/opt/omero/web/OMERO.web/bin/omero config get | grep mapr

shows nothing, so it looks like the mapr config didn't get picked up, which could explain why

DEFAULT_CONFIG = list(mapr_settings.CONFIG.keys())[0]
IndexError: list index out of range

snoopycrimecop · 2019-11-30T00:29:15Z

Conflicting PR. Removed from build OMERO-plugins-push#102. See the console output for more details.
Possible conflicts:

PR Reactivate test-infra with 5.6.0-m2 #56 joshmoore 'Reactivate test-infra with 5.6.0-m2'
- omero_mapr/views.py
- omero_mapr/urls.py

--conflicts

will-moore · 2019-12-02T15:06:11Z

@joshmoore @manics Any idea how to debug the reading of the config file in https://github.com/ome/omero-mapr/blob/master/.omeroci/app-config to see why this might not be read by omero-test-infra?

joshmoore · 2019-12-02T16:32:03Z

I updated .omeroci/app-config to print "XXX". I found a "config.xml not found issue". (This would have turned into a hard-failure with your recently merged PR). ome/omero-test-infra#52 fixes the issue. With that PR, however, I'm seeing:

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1050, in <module>
    'DEPRECATED_SETTINGS_MAPPINGS')
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omeroweb/settings.py", line 1039, in process_custom_settings
    (global_name, key, global_value, e.message, description))
AttributeError: 'JSONDecodeError' object has no attribute 'message'
+ clean_up

snoopycrimecop · 2019-12-03T00:35:08Z

Conflicting PR. Removed from build OMERO-plugins-push#105. See the console output for more details.
Possible conflicts:

PR Reactivate test-infra with 5.6.0-m2 #56 joshmoore 'Reactivate test-infra with 5.6.0-m2'
- omero_mapr/views.py
- omero_mapr/urls.py

--conflicts

will-moore · 2019-12-10T13:00:23Z

With the latest fix to omero-test-infra: ome/omero-test-infra#53 and last commits above, I now see tests running locally but Error on import.
Tried importing to another server (don't know login for docker-compose omero):

$ NOCLEAN=true .omero/docker app
...
=============== 12 passed, 1 xpassed, 28 error in 11.74 seconds ================

$ .omero/compose exec web bash
bash-4.2$ cd /opt/omero/web/OMERO.web/
2$ bin/omero import /opt/omero/web/venv3/lib/python3.6/site-packages/omeroweb/webclient/static/webclient/image/table_layout.png
Server: [localhost:4064]idr1-slot2.openmicroscopy.org
Username: [omero-web]user-3
Password:
Created session for user-3@idr1-slot2.openmicroscopy.org:4064. Idle timeout: 10 min. Current group: read-only-1
Traceback (most recent call last):
  File "/opt/omero/web/venv3/bin/omero", line 131, in <module>
    rv = omero.cli.argv()
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1756, in argv
    cli.invoke(args[1:])
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1187, in invoke
    stop = self.onecmd(line, previous_args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1264, in onecmd
    self.execute(line, previous_args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/cli.py", line 1346, in execute
    args.func(args)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/plugins/import.py", line 518, in importer
    self.do_import(command_args, xargs)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/plugins/import.py", line 529, in do_import
    stdout=out, stderr=err)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/java.py", line 140, in popen
    check_java(command)
  File "/opt/omero/web/venv3/lib64/python3.6/site-packages/omero/java.py", line 31, in check_java
    raise Exception("Java could not be found. (Executable=%s)" % command[0])
Exception: Java could not be found. (Executable=java)

Don't know if this is the same as errors in tests:

        out, err = popen.communicate()
        rc = popen.wait()
        if rc != 0:
            raise Exception("import failed: [%r] %s\n%s\n%s" % (
>               args, rc, out, err))
E           Exception: import failed: [['/opt/omero/web/venv3/bin/python', './bin/omero', '-s', 'omero', '-k', '9084df87-51f5-4922-8cc2-b707882f92d9', '-p', '4064', 'import', '--skip', 'all', '--output', 'legacy', '--', path('/opt/omero/web/OMERO.web/var/omero/tmp/omero_omero-web/336/import_fake_file_01zyg1mji&&plates=1&plateAcqs=1&plateCols=6&plateRows=1&fields=1.fake')]] 1
E           b''
E           b'  File "./bin/omero", line 2\n    . /opt/omero/web/venv3/bin/activate\n    ^\nSyntaxError: invalid syntax\n'

manics · 2019-12-10T13:07:14Z

Which version of the omero-web docker image are you running?

will-moore · 2019-12-10T13:32:37Z

@manics Don't know. Whatever is specified by omero-test-infra.

will-moore · 2019-12-10T13:52:40Z

Travis is green. 😄 I assume the tests are running and passing (but don't see them mentioned in the console).

will-moore · 2019-12-10T15:12:29Z

Going to tidy up and open another clean PR

joshmoore and others added 3 commits November 21, 2019 17:49

Reactivate test-infra with 5.6.0-m2

d698b8b

__init__.py imports urls.py to test import

4f2b693

wrap import urls in try except

18c9a64

will-moore force-pushed the reactivate-infra branch from f589314 to 18c9a64 Compare November 28, 2019 15:31

import omero.clients before ApiException

3c7bc71

will-moore added 2 commits November 28, 2019 16:20

Remove usage of ApiUsageException

2ee8505

Remove usage of ServerError (import failing)

9c18af9

will-moore added 2 commits November 28, 2019 16:45

Add 'DJANGO_SETTINGS_MODULE' to env in setup.py

f88898b

flake8 fix

8794ffb

joshmoore mentioned this pull request Nov 28, 2019

Reactivate test-infra with 5.6.0-m2 #56

Closed

print config to debug

3f2dd05

joshmoore mentioned this pull request Dec 2, 2019

Pass OMERODIR to app-config ome/omero-test-infra#52

Merged

will-moore mentioned this pull request Dec 2, 2019

Fix handling of exceptions in process_custom_settings ome/omero-web#94

Merged

will-moore mentioned this pull request Dec 6, 2019

Fix app-config omero.web.apps formatting ome/omero-test-infra#53

Merged

will-moore added 2 commits December 10, 2019 12:53

Ignore list(mapr_settings.CONFIG.keys())[0] IndexError on test import

ca2b69c

Fix test_config.py - now PASSING

26f3743

will-moore closed this Dec 10, 2019

will-moore mentioned this pull request Dec 10, 2019

Reactivate test infra #58

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reactivate infra #57

Reactivate infra #57

will-moore commented Nov 28, 2019 •

edited

Loading

will-moore commented Nov 28, 2019 •

edited

Loading

will-moore commented Nov 28, 2019 •

edited

Loading

will-moore commented Nov 28, 2019

will-moore commented Nov 28, 2019

snoopycrimecop commented Nov 29, 2019

will-moore commented Nov 29, 2019

will-moore commented Nov 29, 2019

snoopycrimecop commented Nov 30, 2019

will-moore commented Dec 2, 2019

joshmoore commented Dec 2, 2019

snoopycrimecop commented Dec 3, 2019

will-moore commented Dec 10, 2019

manics commented Dec 10, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

Reactivate infra #57

Reactivate infra #57

Conversation

will-moore commented Nov 28, 2019 • edited Loading

will-moore commented Nov 28, 2019 • edited Loading

will-moore commented Nov 28, 2019 • edited Loading

will-moore commented Nov 28, 2019

will-moore commented Nov 28, 2019

snoopycrimecop commented Nov 29, 2019

will-moore commented Nov 29, 2019

will-moore commented Nov 29, 2019

snoopycrimecop commented Nov 30, 2019

will-moore commented Dec 2, 2019

joshmoore commented Dec 2, 2019

snoopycrimecop commented Dec 3, 2019

will-moore commented Dec 10, 2019

manics commented Dec 10, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

will-moore commented Dec 10, 2019

will-moore commented Nov 28, 2019 •

edited

Loading

will-moore commented Nov 28, 2019 •

edited

Loading

will-moore commented Nov 28, 2019 •

edited

Loading