Rule UndefinedJinjaVariablesRule is unusably slow with many DAGs

[jsmodic]

Apache Airflow version:
Airflow 1.10.14
Checker 1.1.0

What happened:

Running "airflow check_upgrade" will basically then work for at least 20+ hours on UndefinedJinjaVariablesRule at 100% CPU (that is until I killed it). I have a folder of 54 DAGs, most of them are fairly complicated.

Stack trace when I killed it:

 File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/checker.py", line 101, in <module>
    __main__()
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/checker.py", line 97, in __main__
    args.func(args)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/checker.py", line 88, in run
    all_problems = check_upgrade(formatter, rules)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/checker.py", line 37, in check_upgrade
    rule_status = RuleStatus.from_rule(rule)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/problem.py", line 44, in from_rule
    result = rule.check()
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/rules/undefined_jinja_varaibles.py", line 141, in check
    dag_messages = self.iterate_over_dag_tasks(dag)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/rules/undefined_jinja_varaibles.py", line 128, in iterate_over_dag_tasks
    error_messages = self.iterate_over_template_fields(task)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/rules/undefined_jinja_varaibles.py", line 114, in iterate_over_template_fields
    task, content, context
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/upgrade/rules/undefined_jinja_varaibles.py", line 96, in _render_task_content
    renderend_content = task.render_template(content, context)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/models/baseoperator.py", line 755, in render_template
    return jinja_env.from_string(content).render(**context)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/environment.py", line 941, in from_string
    return cls.from_code(self, self.compile(source), globals, None)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/environment.py", line 629, in compile
    source = self._generate(source, name, filename, defer_init=defer_init)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/environment.py", line 592, in _generate
    optimized=self.optimized,
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 87, in generate
    generator.visit(node)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/visitor.py", line 32, in visit
    return f(node, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 774, in visit_Template
    self.blockvisit(node.body, frame)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 381, in blockvisit
    self.visit(node, frame)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/visitor.py", line 32, in visit
    return f(node, *args, **kwargs)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 1429, in visit_Output
    self.writeline("yield " + val)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 402, in writeline
    self.write(x)
  File "/home/ubuntu/.local/lib/python3.6/site-packages/jinja2/compiler.py", line 396, in write
    self._new_lines = 0

I can just ignore that rule no problem and it finishes in a few seconds, but I imagine it's going to be quite confusing to anyone else that gets into this situation where it looks stuck without digging into the source.

[kaxil]

For now, users should just ignore this check until a fix comes out in Airflow Upgrade Check - 1.2.0 :

airflow upgrade_check -i UndefinedJinjaVariablesRule