Custom SQL filters not working with Virtual (SQL) dataset containing Jinja template

[grumpy-miner]

When using Custom SQL filter in graphs where dataset is Virtual (SQL) dataset containing Jinja template as SQL, superset raises an error.

Expected results

Graph is rendered properly

Actual results

Superset raises 'NoneType' object has no attribute 'upper'

Traceback (most recent call last):
  File "/app/superset/viz.py", line 541, in get_df_payload
    df = self.get_df(query_obj)
  File "/app/superset/viz.py", line 277, in get_df
    self.results = self.datasource.query(query_obj)
  File "/app/superset/connectors/sqla/models.py", line 1438, in query
    query_str_ext = self.get_query_str_extended(query_obj)
  File "/app/superset/connectors/sqla/models.py", line 764, in get_query_str_extended
    sqlaq = self.get_sqla_query(**query_obj)
  File "/app/superset/connectors/sqla/models.py", line 1143, in get_sqla_query
    tbl = self.get_from_clause(template_processor)
  File "/app/superset/connectors/sqla/models.py", line 793, in get_from_clause
    from_sql = self.get_rendered_sql(template_processor)
  File "/app/superset/connectors/sqla/models.py", line 814, in get_rendered_sql
    sql = template_processor.process_template(sql)
  File "/app/superset/jinja_context.py", line 426, in process_template
    return template.render(context)
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 2, in top-level template code
  File "/usr/local/lib/python3.7/site-packages/jinja2/sandbox.py", line 462, in call
    return __context.call(__obj, *args, **kwargs)
  File "/app/superset/jinja_context.py", line 323, in safe_proxy
    return_value = func(*args, **kwargs)
  File "/app/superset/jinja_context.py", line 211, in filter_values
    filters = self.get_filters(column, remove_filter)
  File "/app/superset/jinja_context.py", line 300, in get_filters
    op: str = flt["operator"].upper() if "operator" in flt else None
AttributeError: 'NoneType' object has no attribute 'upper'

Screenshots

How to reproduce the bug

1. Create table in database (in my case Postgres 13)

CREATE TABLE test_table (
  id INTEGER,
  some_text TEXT,
  some_date TIMESTAMPTZ,
  some_integer INTEGER
);

2. Create Virtual dataset
   schema: public
   dataset_name: test_table
   SQL:

select * from test_table
{% if filter_values('tag')|length  %}
where 1=1
{% endif %}

3. Create new chart with visualisation type Time-series Bar Chart
4. Set Group By: some_text
5. Set Metrics: count(*)
6. Set Filters as Custom SQL with statement where 1=1
7. Click Run

Environment

• superset version: Superset 0.0.0dev
• Superset [image: apache/superset:latest-dev, pulled 2021-08-19]
• python version: Python 3.7.9
• feature flags active:
  • ENABLE_TEMPLATE_PROCESSING

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

In url parameters of the chart there is parameter operator set to null

In file superset/jinja_context.py line 300

op: str = flt["operator"].upper() if "operator" in flt else None

logic doesn't assume that flt dictionary can have operator with value None and that's what causes the problem

[villebro]

fixed by#16371