Boolean field in elasticsearch index mapping causes query to fail

[stankovic-marko]

Issue Summary

This bug occurs when querying Elasticsearch index which has a boolean field as its mapping. Here is a mapping example:

{
   "my_index":{
      "mappings":{
         "dynamic_templates":[
            {
               ...
            }
         ],
         "date_detection":false,
         "properties":{
            ...
         }
      }
   }
}

The field that causes an error in this example is date_detection.

Steps to Reproduce

1. Add index mapping that has boolean field as a value (for example date_detection).
2. Run any search query on index with mapping in question.
3. Get this error: Error running query: argument of type 'bool' is not iterable

Technical details:

• Redash Version: v10.1.0 and preview version
• Browser/OS: Firefox, Ubuntu v22.04
• How did you install Redash: running setup.sh script

Here is also a log retrieved from docker compose logs:

adhoc_worker-1      | [2024-04-22 13:14:19,403][PID:119][INFO][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=a2b1c322-fcb9-4923-ba0c-6b186013562e job=execute_query state=executing_query query_hash=12a4edf85528ecb9a65594f625d13ee3 type=elasticsearch ds_id=1 job_id=a2b1c322-fcb9-4923-ba0c-6b186013562e queue=queries query_id=1 username=*
adhoc_worker-1      | [2024-04-22 13:14:19,414][PID:119][WARNING][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=a2b1c322-fcb9-4923-ba0c-6b186013562e Unexpected error while running query:
adhoc_worker-1      | Traceback (most recent call last):
adhoc_worker-1      |   File "/app/redash/tasks/queries/execution.py", line 182, in run
adhoc_worker-1      |     data, error = query_runner.run_query(annotated_query, self.user)
adhoc_worker-1      |   File "/app/redash/query_runner/elasticsearch.py", line 449, in run_query
adhoc_worker-1      |     mappings, error = self._get_query_mappings(mapping_url)
adhoc_worker-1      |   File "/app/redash/query_runner/elasticsearch.py", line 132, in _get_query_mappings
adhoc_worker-1      |     if "properties" not in index_mappings["mappings"][m]:
adhoc_worker-1      | TypeError: argument of type 'bool' is not iterable
adhoc_worker-1      | [2024-04-22 13:14:19,415][PID:119][INFO][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=a2b1c322-fcb9-4923-ba0c-6b186013562e job=execute_query query_hash=12a4edf85528ecb9a65594f625d13ee3 ds_id=1 data_length=None error=[argument of type 'bool' is not iterable]
adhoc_worker-1      | [2024-04-22 13:14:19,418][PID:119][INFO][rq.worker] queries: Job OK (a2b1c322-fcb9-4923-ba0c-6b186013562e)
adhoc_worker-1      | [2024-04-22 13:14:19,418][PID:119][INFO][rq.worker] Result is kept for 43200 seconds

Here is where the error occurs:

redash/redash/query_runner/elasticsearch.py

Lines 129 to 143 in 6c68b48

	for index_name in mappings_data:
	index_mappings = mappings_data[index_name]
	for m in index_mappings.get("mappings", {}):
	if "properties" not in index_mappings["mappings"][m]:
	continue
	for property_name in index_mappings["mappings"][m]["properties"]:
	property_data = index_mappings["mappings"][m]["properties"][property_name]
	if property_name not in mappings:
	property_type = property_data.get("type", None)
	if property_type:
	if property_type in ELASTICSEARCH_TYPES_MAPPING:
	mappings[property_name] = ELASTICSEARCH_TYPES_MAPPING[property_type]
	else:
	mappings[property_name] = TYPE_STRING
	# raise Exception("Unknown property type: {0}".format(property_type))

Possible solution

mappings = {}
for index_name in mappings_data:
    index_mappings = mappings_data[index_name]
    for m in index_mappings.get("mappings", {}):
        if not isinstance(index_mappings["mappings"][m], dict):
            continue
        if "properties" not in index_mappings["mappings"][m]:
            continue
        for property_name in index_mappings["mappings"][m]["properties"]:
            property_data = index_mappings["mappings"][m]["properties"][property_name]
            if property_name not in mappings:
                property_type = property_data.get("type", None)
                if property_type:
                    if property_type in ELASTICSEARCH_TYPES_MAPPING:
                        mappings[property_name] = ELASTICSEARCH_TYPES_MAPPING[property_type]
                    else:
                        mappings[property_name] = TYPE_STRING

[AndrewChubatiuk]

Hi @stankovic-marko
Could you please submit a PR?

[stankovic-marko]

Hi @AndrewChubatiuk
I have submitted a PR.