Empty arrays not returned

[djstrong]

Elasticsearch version (bin/elasticsearch --version): 8.6.2

elasticsearch-py version (elasticsearch.__versionstr__): 8.6.2

Please make sure the major version matches the Elasticsearch server you are running.

Description of the problem including expected versus actual behavior:
I have a document:

{
  "a": {
    "b": []
  }
}

and I search it asking for fields: {"fields": ["a.b"]}. If b is empty array (as above) then this field in response is missing, but when this array is not empty then it is returned.
I expect that in the first case the empty array will be returned.

[pquentin]

Hello! Sorry for not getting back to you earlier. Per https://www.elastic.co/guide/en/elasticsearch/reference/current/array.html:

An array may contain null values, which are either replaced by the configured null_value or skipped entirely. An empty array [] is treated as a missing field — a field with no values.

As a result, you have two options:

1. If you want to get the data exactly as you indexed it, you should use {"source": ["a.b"]}
2. If you want to keep using fields, you can configure null_value as mentioned in the docs

Here's a snippet comparing the two approaches:

import json

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

es.options(ignore_status=404).indices.delete(index="emptyarray")
es.indices.create(
    index="emptyarray",
    mappings={
        "properties": {
            "a": {"properties": {"b": {"type": "keyword", "null_value": "NULL"}}},
            "c": {"type": "keyword"},
        }
    },
)

es.create(
    index="emptyarray",
    id="1",
    body={"a": {"b": []}, "c": "d"},
    refresh=True,
)


es.create(
    index="emptyarray",
    id="2",
    body={"a": {"b": None}, "c": "d"},
    refresh=True,
)


resp = es.search(index="emptyarray", source=["a.b"])
print(json.dumps(resp.body["hits"]["hits"], indent=2))

# Prints:
# [
#   {
#     "_index": "emptyarray",
#     "_id": "1",
#     "_score": 1.0,
#     "_source": {
#       "a": {
#         "b": []
#       }
#     }
#   },
#   {
#     "_index": "emptyarray",
#     "_id": "2",
#     "_score": 1.0,
#     "_source": {
#       "a": {
#         "b": null
#       }
#     }
#   }
# ]

resp = es.search(index="emptyarray", fields=["a.b"], source=False)
print(json.dumps(resp.body["hits"]["hits"], indent=2))

# Prints:
# [
#   {
#     "_index": "emptyarray",
#     "_id": "1",
#     "_score": 1.0
#   },
#   {
#     "_index": "emptyarray",
#     "_id": "2",
#     "_score": 1.0,
#     "fields": {
#       "a.b": [
#         "NULL"
#       ]
#     }
#   }
# ]

[djstrong]

Thank you!