2.9.1 incompatibility with flask admin

[bolkedebruin]

We have jinja2 in Airflow and it throws an error that seems tied to our usage of flask-admin:

======================================================================
ERROR: test_dag_views (tests.WebUiTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/apache/incubator-airflow/tests/core.py", line 1423, in test_dag_views
    '/admin/airflow/graph?dag_id=example_bash_operator')
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/werkzeug/test.py", line 778, in get
    return self.open(*args, **kw)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/testing.py", line 127, in open
    follow_redirects=follow_redirects)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/werkzeug/test.py", line 751, in open
    response = self.run_wsgi_app(environ, buffered=buffered)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/werkzeug/test.py", line 668, in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/werkzeug/test.py", line 871, in run_wsgi_app
    app_rv = app(environ, start_response)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1994, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
    response = self.handle_exception(e)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask_login.py", line 755, in decorated_view
    return func(*args, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/airflow/www/utils.py", line 221, in view_func
    return f(*args, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/airflow/www/utils.py", line 125, in wrapper
    return f(*args, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/airflow/www/views.py", line 1438, in graph
    edges=json.dumps(edges, indent=2),)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask_admin/base.py", line 308, in render
    return render_template(template, **kwargs)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/build/apache/incubator-airflow/airflow/www/templates/airflow/graph.html", line 19, in top-level template code
    {% import 'admin/lib.html' as lib with context %}
  File "/home/travis/build/apache/incubator-airflow/.tox/py27-cdh-airflow_backend_sqlite/lib/python2.7/site-packages/jinja2/environment.py", line 551, in _compile
    return compile(source, filename, 'exec')
SyntaxError: duplicate argument 'l_1_caller' in function definition (lib.html, line 255)

[ThiefMaster]

Looks like a regression. Until it's fixed I'd recommend requiring jinja2<2.9 in your package requirements

[bolkedebruin]

We are now. Thanks.

[mitsuhiko]

Would it be possible to get a minimal repro case for this?

[mitsuhiko]

Found a minimal repo. Interesting that this worked before:

{% macro x(caller=none) -%}
    [{% if caller %}{{ caller() }}{% endif %}]
{%- endmacro %}

{{ x() }}
{% call x() %}aha!{% endcall %}

Going to investigate what this did.

[ThiefMaster]

isn't that caller=none arg pointless anyway? caller is already none (or undefined?) when calling a macro without {% call %}

[mitsuhiko]

@ThiefMaster it's undefined and yes. I assume the correct behavior will be to just silently ignore caller here. However I wonder if you could set it to a different default before.

[mitsuhiko]

Eg. Could you do {% macro x(caller=42) %}...{% endmacro %}.

[ThiefMaster]

sounds like something that should at least trigger a warning

[mitsuhiko]

So you could define a default in the past but only if caller was the last argument I think. If you did {% macro x(caller, foo, bar) %} then it would fail when used with {% call %}.

[mitsuhiko]

So turns out this is common. https://github.com/search?q=%22%7B%25+macro+%22+%22caller%3Dnone%22&type=Code&utf8=%E2%9C%93

[mitsuhiko]

Not sure if that change is good enough to cover all cases but this is a change I am willing to make. This at least gives it a clear behavior now.

[bolkedebruin]

That was quick! Thanks, will verify our own code whether we do something "unexpected".