Source filtering ("_source":false) don't work in collapse->inner_hits

[Chakrygin]

Elasticsearch version: 5.3.0

Plugins installed: []

JVM version:
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

OS version: Windows 10

Description of the problem including expected versus actual behavior:
Disabling "_source" retrieval (with setting it to false) don't work if used with collapse->inner_hits

Steps to reproduce:

1. Test data:

POST http://localhost:9200/_bulk

{"index":{"_index":"test","_type":"test","_id":"1"}}
{"id":1,"group_id":4,"name":"one"}
{"index":{"_index":"test","_type":"test","_id":"2"}}
{"id":2,"group_id":4,"name":"two"}
{"index":{"_index":"test","_type":"test","_id":"3"}}
{"id":3,"group_id":4,"name":"three"}

2. Test query:

POST http://localhost:9200/test/test/_search

{
	"query" : {
		"match_all" : { }
	},
	
	"_source" : false,
	
	"collapse" : {
		"field" : "group_id",
		"inner_hits" : {
			"name" : "some_name",
			"_source" : false
		}
	}
}

Describe the feature:

The response still contains "_source" elements:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 3,
    "max_score": null,
    "hits": [
      {
        "_index": "test",
        "_type": "test",
        "_id": "2",
        "_score": 1,
        "fields": {
          "group_id": [
            4
          ]
        },
        "inner_hits": {
          "some_name": {
            "hits": {
              "total": 3,
              "max_score": 1,
              "hits": [
                {
                  "_index": "test",
                  "_type": "test",
                  "_id": "2",
                  "_score": 1,
                  "_source": {
                    "id": 2,
                    "group_id": 4,
                    "name": "two"
                  }
                },
                {
                  "_index": "test",
                  "_type": "test",
                  "_id": "1",
                  "_score": 1,
                  "_source": {
                    "id": 1,
                    "group_id": 4,
                    "name": "one"
                  }
                },
                {
                  "_index": "test",
                  "_type": "test",
                  "_id": "3",
                  "_score": 1,
                  "_source": {
                    "id": 3,
                    "group_id": 4,
                    "name": "three"
                  }
                }
              ]
            }
          }
        }
      }
    ]
  }
}

[polyfractal]

Thanks for the detailed report! I can reproduce this on 5.x and master.

Although this is in the context of field collapsing, I suspect it's ultimately an inner_hits problem similar to #21312. I'm going to close this issue and add a link back to it from #21312 so that we can keep all the details about this bug in one place. :)

[nerophon]

As of 6.5.1, this particular sub-issue is still reproducible despite the parent issue being closed.

DELETE products
PUT /products
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "_doc": {
      "properties": {
        "id": {
          "type": "integer"
        },
        "description": {
          "type": "text"
        },
        "clicks": {
          "type": "integer"
        },
        "views": {
          "type": "integer"
        }
      }
    }
  }
}
POST products/_doc/1
{
    "id" : 1,
    "description": "teste 1",
    "clicks": 123,
    "views": 1
}

POST products/_doc/2
{
    "id" : 1,
    "description": "teste 2",
    "clicks": 1,
    "views": 200
}

POST products/_doc/3
{
    "id" : 3,
    "description": "teste 3",
    "clicks": 1,
    "views": 200
}

POST products/_search
{
    "from": 0,
    "size": 10,
    "_source": false,
    "collapse" : {
        "field" : "id",
        "inner_hits" : {
          "_source": false,
          "name" : "product",
            "docvalue_fields" : [
              {
                "field": "id",
                "format": "use_field_mapping"
              }]
        }
    }
}

Perhaps this is specific to field-collapsing, or the main issue didn't resolve all parts of the problem. Notably, using the full-object representation of _source does succeed in delivering a result with an empty source field, i.e. "_source" : { },. To achieve this I used:

          "_source": {
            "includes": [ "" ]
          },

[elasticmachine]

Pinging @elastic/es-search