Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Support GAE #38

Closed
fread2281 opened this Issue · 15 comments

4 participants

@fread2281

Some import fails when run on GAE's sandbox and ys.path.append(os.path.join(os.path.dirname(__file__), 'venv/lib/python2.7/site-packages')) is used to add all extra modules:

ERROR    2012-11-04 20:36:44,336 cgi.py:121] Traceback (most recent call last):
  File "/Users/cartercharbonneau/dndapp/run.py", line 3, in <module>
    from dndapp import app
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1928, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1792, in FindAndLoadModule
    description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1735, in LoadModuleRestricted
    description)
  File "/Users/cartercharbonneau/dndapp/dndapp/__init__.py", line 7, in <module>
    cache = Cache(app, False)
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask_cache/__init__.py", line 64, in __init__
    self.init_app(app)
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask_cache/__init__.py", line 98, in init_app
    self._set_cache()
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask_cache/__init__.py", line 106, in _set_cache
    cache_obj = import_string(import_me)
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/werkzeug/utils.py", line 422, in import_string
    __import__(modname)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1928, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1792, in FindAndLoadModule
    description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 719, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1735, in LoadModuleRestricted
    description)
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask_cache/backends.py", line 1, in <module>
    from werkzeug.contrib.cache import (NullCache, SimpleCache, MemcachedCache,
  File "/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask/exthook.py", line 86, in load_module
    raise ImportError('No module named %s' % fullname)
ImportStringError: import_string() failed for 'flask.ext.cache.backends.gaememcached'. Possible reasons are:

- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;

Debugged import:

- 'flask' found in '/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask/__init__.pyc'.
- 'flask.ext' found in '/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask/ext/__init__.pyc'.
- 'flask.ext.cache' found in '/Users/cartercharbonneau/dndapp/venv/lib/python2.7/site-packages/flask_cache/__init__.pyc'.
- 'flask.ext.cache.backends' not found.

Original exception:

ImportError: No module named flask.ext.cache.werkzeug
@thadeusb
Owner

I do not have a way to test GAE so you will have to figure the fix out and we can include it.

It appears that its attempting to import werkzeug as a local module as if you do not have werkzeug installed in your path.

@LeonB

I believe I'm having something similar:

ImportError: No module named flask.ext.cache.jinja2

Up to version 0.7 of Flask-Cache it works fine. 0.8 & 0.9 are throwing the above error.

@thadeusb
Owner

At version 0.8 the package was restructured to support the new flask extension import mechanism, possible collision with the way Flask implements its import?

@irvingpop

I'm having the same issue with Flask-Cache > 0.8 just like @LeonB

I am using https://github.com/kamalgill/flask-appengine-template, which implements pkg_resources. Reverting to 0.7.x in flaskext/ and it works fine.

The issue is that flask/exthook.py is getting confused by the imports in jinja2ext.py.

Any idea what's going wrong?

@thadeusb
Owner

Please see if the AppEngineImport branch fixes the problem.

https://github.com/thadeusb/flask-cache/tree/AppEngineImport

@irvingpop

@thadeusb, thanks for taking the time on this one. I'm still seeing the same error using the AppEngineImport branch pulled today.

The line in jinja2ext.py:
from jinja2 import nodes

results in

 File "/Users/irving/personal/BeerMenu/beermenu3/flask/flask/exthook.py", line 86, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.cache.jinja2

Why would flask/exthook.py try to look for jinja2 underneath flask.ext.cache? it makes no sense.

Full Traceback:

 from flask.ext.cache.jinja2ext import CacheExtension, \
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1766, in load_module
    return self.FindAndLoadModule(submodule, fullname, search_path)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1630, in FindAndLoadModule
    description)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 692, in Decorate
    return func(self, *args, **kwargs)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1577, in LoadModuleRestricted
    description)
  File "/Users/irving/personal/BeerMenu/beermenu3/flask/flaskext/cache/jinja2ext.py", line 31, in <module>
    from jinja2 import nodes
  File "/Users/irving/personal/BeerMenu/beermenu3/flask/flask/exthook.py", line 86, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named flask.ext.cache.jinja2
@thadeusb
Owner

Might need to file a bug in flask.

I comment out line 86 where it raises the import error and everything seems to be working fine.

This seems to be the problem, the python docs state that a meta_path object should return None if it was unable to import a specific module. http://docs.python.org/2/library/sys.html#sys.meta_path

This allows the import system to continue and search sys.path as normal.

@irvingpop

Thanks for the excellent diagnosis. 3 questions for you:

  1. Are we still sure this is an AppEngine thing, or could it affect other environments as well?
  2. Would you like me to file the bug report in Flask, or do you want to do that?
  3. So I can summarize the issue correctly, let me know if I have this right or not:

Summary: Flask ExtensionImporter class (flask.ext) is getting in the way when flask extensions import external modules (possibly AppEngine only)

What is happening:
1. flask extension (ex: Flask-Cache) imports a module (ex: "from jinja2 import nodes")
2. in the extension's context, ExtensionImporter class has added its own path (flask.ext) to sys.meta_path
3. When ExtensionImporter load_module() is run, it fails to load the external module (with an ImportError?) and thus bombs

  • this may be caused by AppEngine's dev_appserver_import_hook.py, line 1577, in LoadModuleRestricted
  • The application will run normally if you comment out line 86 in flask/exthook.py "raise ImportError('No module named %s' % fullname)"
  • or by reverting to the the old flaskext path
@thadeusb
Owner

It may affect other environments. I have a sneaky suspicion that it is due to the way that AppEngine has its own importer.

You can file the bug, since you are the one who can check to make sure it gets fixed and continues to work in AppEngine.

Looks like you have the problem down right. I am curious, does this problem appear when using other flask extensions that use the new flaskext path mechanism?

@irvingpop

OK, I'll file the bug.

I haven't tried any other flask.ext plugins yet, mainly because I didn't need any of them. Are there any you'd suggest I try, or a reference/example flask.ext plugin that imports external modules?

@irvingpop

issue filed here: mitsuhiko/flask#662

@thadeusb
Owner

Can you please see if the latest AppEngineImport branch works for you?

I believe the work-around for now is to use relative imports instead of absolute. I don't like it as its not as explicit/clean, but it seems to work.

@irvingpop

It works now! thank you!

@fread2281

I've moved away from GAE, but that's good news, closing this now.

@fread2281 fread2281 closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.