mako 1.0.1 is incompatible with Python 3.5

[sqlalchemy-bot]

Migrated issue, originally created by Barry Warsaw (@warsaw)

In Ubuntu we've enabled Python 3.5 as a supported version. mako 1.0.0 and 1.0.1 are incompatible with it and fail to build because of the use of a new API in the ast module. Here's a paste of the test failures.

======================================================================
ERROR: test_argument_list (test.test_ast.AstParseTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/«PKGBUILDDIR»/test/test_ast.py", line 274, in test_argument_list
    "context.get('lala')", **exception_kwargs)
  File "/«PKGBUILDDIR»/mako/ast.py", line 61, in __init__
    f.visit(expr)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 287, in generic_visit
    self.visit(item)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 289, in generic_visit
    self.visit(value)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 186, in visit_Tuple
    self.listener.args.append(ExpressionGenerator(n).value())
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 229, in __init__
    self.generator.visit(astnode)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 649, in visit_Call
    if node.starargs is not None:
    lineno=line, pos=pos)
  File "/«PKGBUILDDIR»/mako/lexer.py", line 132, in append_node
    node = nodecls(*args, **kwargs)
  File "/«PKGBUILDDIR»/mako/parsetree.py", line 179, in __init__
    self.escapes_code = ast.ArgumentList(escapes, **self.exception_kwargs)
  File "/«PKGBUILDDIR»/mako/ast.py", line 61, in __init__
    f.visit(expr)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 287, in generic_visit
    self.visit(item)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 289, in generic_visit
    self.visit(value)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 186, in visit_Tuple
    self.listener.args.append(ExpressionGenerator(n).value())
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 229, in __init__
    self.generator.visit(astnode)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 649, in visit_Call
    if node.starargs is not None:
AttributeError: 'Call' object has no attribute 'starargs'

======================================================================
ERROR: test_escapes_html_tags (test.test_template.EncodingTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/«PKGBUILDDIR»/test/test_template.py", line 37, in test_escapes_html_tags
    x.render()
  File "/«PKGBUILDDIR»/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/«PKGBUILDDIR»/mako/runtime.py", line 803, in _render
    **_kwargs_for_callable(callable_, data))
  File "/«PKGBUILDDIR»/mako/runtime.py", line 835, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/«PKGBUILDDIR»/mako/runtime.py", line 860, in _exec_template
    callable_(context, *args, **kwargs)
  File "memory:0x7f5427e21da0", line 21, in render_body
Exception: <span style="color:red">Foobar</span>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/«PKGBUILDDIR»/test/test_template.py", line 40, in test_escapes_html_tags
    markup = html_error_template().render(full=False, css=False)
  File "/«PKGBUILDDIR»/mako/exceptions.py", line 373, in html_error_template
    encoding_errors='htmlentityreplace')
  File "/«PKGBUILDDIR»/mako/template.py", line 303, in __init__
    (code, module) = _compile_text(self, text, filename)
  File "/«PKGBUILDDIR»/mako/template.py", line 666, in _compile_text
    generate_magic_comment=template.disable_unicode)
  File "/«PKGBUILDDIR»/mako/template.py", line 647, in _compile
    node = lexer.parse()
  File "/«PKGBUILDDIR»/mako/lexer.py", line 232, in parse
    if self.match_expression():
  File "/«PKGBUILDDIR»/mako/lexer.py", line 393, in match_expression
    lineno=line, pos=pos)
  File "/«PKGBUILDDIR»/mako/lexer.py", line 132, in append_node
    node = nodecls(*args, **kwargs)
  File "/«PKGBUILDDIR»/mako/parsetree.py", line 179, in __init__
    self.escapes_code = ast.ArgumentList(escapes, **self.exception_kwargs)
  File "/«PKGBUILDDIR»/mako/ast.py", line 61, in __init__
    f.visit(expr)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 287, in generic_visit
    self.visit(item)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 279, in visit
    return self.generic_visit(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 289, in generic_visit
    self.visit(value)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 186, in visit_Tuple
    self.listener.args.append(ExpressionGenerator(n).value())
  File "/«PKGBUILDDIR»/mako/pyparser.py", line 229, in __init__
    self.generator.visit(astnode)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 278, in visit
    return f(node)
  File "/«PKGBUILDDIR»/mako/_ast_util.py", line 649, in visit_Call
    if node.starargs is not None:
AttributeError: 'Call' object has no attribute 'starargs'

----------------------------------------------------------------------
Ran 458 tests in 2.559s

FAILED (errors=14, skipped=59)

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

hi Barry -

are there any changes in Python 3.5 to the specification listed here: https://docs.python.org/3/library/ast.html#abstract-grammar ? call is listed like this:

#!


 Call(expr func, expr* args, keyword* keywords,
             expr? starargs, expr? kwargs)

that means may or may not have "starargs" or "kwargs", so we just need to check with hasattr(), is that correct?

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

don't have 3.5 built at the moment, if you want to try this patch:

diff --git a/mako/_ast_util.py b/mako/_ast_util.py
index 604b090..cc298d5 100644
--- a/mako/_ast_util.py
+++ b/mako/_ast_util.py
@@ -486,11 +486,11 @@ class SourceGenerator(NodeVisitor):
                 paren_or_comma()
                 self.write(keyword.arg + '=')
                 self.visit(keyword.value)
-            if node.starargs is not None:
+            if getattr(node, "starargs", None):
                 paren_or_comma()
                 self.write('*')
                 self.visit(node.starargs)
-            if node.kwargs is not None:
+            if getattr(node, "kwargs", None):
                 paren_or_comma()
                 self.write('**')
                 self.visit(node.kwargs)
@@ -652,11 +652,11 @@ class SourceGenerator(NodeVisitor):
             write_comma()
             self.write(keyword.arg + '=')
             self.visit(keyword.value)
-        if node.starargs is not None:
+        if getattr(node, "starargs", None):
             write_comma()
             self.write('*')
             self.visit(node.starargs)
-        if node.kwargs is not None:
+        if getattr(node, "kwargs", None):
             write_comma()
             self.write('**')
             self.visit(node.kwargs)

[sqlalchemy-bot]

Barry Warsaw (@warsaw) wrote:

Hi Mike,

Yes, that looks like it fixed the build failure. Thanks! I'll upload 1.0.1 with this patch to Debian and Ubuntu. One other thing to keep in mind for 3.5 is that inspect.getargspec() is deprecated. You get a DeprecationWarning when running the test suite under 3.5, but it doesn't break the build so I'm not going to worry about it. :)

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

• Repair some calls within the ast module that no longer work on Python3.5;
  additionally replace the use of inspect.getargspec() under
  Python 3 (seems to be called from the TG plugin) to avoid deprecation
  warnings.
  fixes mako 1.0.1 is incompatible with Python 3.5 #250

→ 4707bb0

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed

[sqlalchemy-bot]

Barry Warsaw (@warsaw) wrote:

On Aug 26, 2015, at 05:58 PM, Mike Bayer wrote:

One thing I still can't figure out. When I build the package, we run $python setup.py test with the various versions of supported Pythons. On Debian,
this works fine for 2.7 and 3.4, but on Ubuntu, for all Python 3 versions,
setup.py test seems to require the python3-funcsigs package. I've grepped
the mako source and can't find any reference to it, so I'm stumped. Any
thoughts?

--- you can reply above this line ---

Issue 250: mako 1.0.1 is incompatible with Python 3.5
https://bitbucket.org/zzzeek/mako/issues/250/mako-101-is-incompatible-with-python-35

Mike Bayer:

• Repair some calls within the ast module that no longer work on Python3.5;
  additionally replace the use of inspect.getargspec() under
  Python 3 (seems to be called from the TG plugin) to avoid deprecation
  warnings.
  fixes mako 1.0.1 is incompatible with Python 3.5 #250

→ 4707bb0

Changes:

--

Unwatch this issue to stop receiving email updates:
https://bitbucket.org/api/1.0/zzzeek/mako/issue/250/unwatch/warsaw/04ee750dadfa2543838ab43a0d8f4f73e6989c8f/

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

hmmm, well when i google python3-funcsigs most of the hits seem to be ubuntu-specific. It looks like this package provides the "signature" series of functions that Python 3.4 seems to say that "getfullargspec()" now builds on top of. So perhaps something in Ubuntu. Does a command like "pip freeze" illustrate this package as a dependency?