Urls import handling

[will-moore]

This change means that failure to import module.urls.py for OMERO.web apps results in a failure of OMERO.web to run. Failure to import the module itself has always resulted in a failure of OMERO.web to run, but before this PR, import of module.urls.py fails silently.
This has meant that an app could be "installed" and OMERO.web appear to run OK, but all of the app's URLS are 404 or that reverse(url_name) fails.

This PR implements the behaviour that I manually applied each time I was working to port an OMERO.web app to python3 where I needed to see import failures at start-time, and not silently result in 404s.
Hopefully this will also help fixing failure of reverse("mapr_config") etc in integration tests at ome/omero-mapr#54 by causing a failure at an earlier point.

To test, run OMERO.web locally, install an app (e.g figure, iviewer etc) then add an error that causes an ImportError, either in the urls.py or views.py and check that OMERO.web fails with ImportError. Could also use a pre-python3 version of figure or iviewer etc.

[will-moore]

As discussed: strategy of failing on import is 👍 but would be nice to have more friendly error messages.

[will-moore]

With that last commit, you now see this for failed apps (weberror)

Show error

...
Performing system checks...

Failed to import omero_weberror.urls
Please check if the app installed and the versions of the app and OMERO.web are compatible
            
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x10fa76e60>
Traceback (most recent call last):
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 228, in wrapper
    fn(*args, **kwargs)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 124, in inner_run
    self.check(display_num_errors=True)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/management/base.py", line 359, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/management/base.py", line 346, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/checks/registry.py", line 81, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/checks/urls.py", line 16, in check_url_config
    return check_resolver(resolver)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/core/checks/urls.py", line 26, in check_resolver
    return check_method()
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/urls/resolvers.py", line 256, in check
    for pattern in self.url_patterns:
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/urls/resolvers.py", line 407, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/utils/functional.py", line 35, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/urls/resolvers.py", line 400, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/willadmin/Desktop/WEB/omero-web/omeroweb/urls.py", line 91, in <module>
    __import__(urlmodule)
  File "/Users/willadmin/Desktop/WEBERROR/omero-weberror/omero_weberror/urls.py", line 26, in <module>
    from django.conf.urls import url, patterns
ImportError: cannot import name 'patterns' from 'django.conf.urls' (/Users/willadmin/Desktop/py3/venv/lib/python3.7/site-packages/django/conf/urls/__init__.py)

[joshmoore]

PR has been in the build for a while now. In general, 👍 for more feedback to the users when something has gone wrong. One minor comment that can likely be addressed and merged with no further testing.

[joshmoore]

Should this be higher than "debug"?

[will-moore]

I don't think so. This just means that an app doesn't have a urls.py. E.g. corsheaders. It's not a failure since we don't expect all apps to have it.
I guess thee is an expectation that omero web apps will have a urls.py, but we have no concrete way of telling which app is an omero-web app.
Perhaps if urlmodule.startswith("ome") we could log at info, or even error? But it's still possible this is not really an error. You're not likely to accidentally forget to add a urls.py?

[joshmoore]

At least not and expect it to work 😄 Thanks for the explanation!