Elasticsearch 6.1.1 - Not possible to specify 'format' in Date Histogram aggregation when used in Composite Aggregation

[byronvoorbach]

Hi all!

On Elasticsearch 6.1.1 I'm getting a:

{
  "error": {
    "root_cause": [
      {
        "type": "parsing_exception",
        "reason": "[composite] failed to parse field [sources]",
        "line": 26,
        "col": 17
      }
    ],
    "type": "parsing_exception",
    "reason": "[composite] failed to parse field [sources]",
    "line": 26,
    "col": 17,
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "[date_histogram] unknown field [format], parser not found"
    }
  },
  "status": 400
}

When trying to choose a format for my date_histogram aggregation, which is placed in a composite aggregation:

GET tickers/_search
{
  "size": 0,
  "aggs": {
    "compos": {
      "composite": {
        "size": 50,
        "sources": [
          {
            "currencyPair": {
              "terms": {
                "field": "currencyPair.raw"
              }
            }
          },
          {
            "exchanges": {
              "terms": {
                "field": "exchange.raw"
              }
            }
          },
          {
            "time" : {
              "date_histogram" : {
                "field": "date",
                "format" : "yyyy-MM-dd--HH:mm:ss.SSSZ",
                "interval": "5m"
              }
            }
          }
        ]
      }
    }
  }
}

Is this a possible bug? Or is it not possible to have a 'key_as_string' when using composite?
Here is the complete code to reproduce:

https://gist.github.com/byronvoorbach/434f84a8cf89f6aab9cb2d9e291a9047

[jimczi]

Is this a possible bug? Or is it not possible to have a 'key_as_string' when using composite?

This is not supported in 6.1, the composite agg returns the raw value but I agree that it should be possible to set a format for date fields in future versions. I marked this issue as a feature request and will update it when a pr is ready.

Thanks for testing the composite aggregation @byronvoorbach !

[byronvoorbach]

Hey @jimczi

Ahh okay! Thanks for your response! I'm enthusiastic about the composite aggregation :D Nice addition
I wouldn't mind working on a PR either if needed :)

[DavidKHE]

Hi,

thank you for the resolution and the support for the format on composite date_histogram.

But, it seems to me that when using in AFTER_KEY. it is ignoring the date key when using formatted.

Is there any need to specify this differently?

Thanks,
David

[jimczi]

@DavidKHE this issue is closed but you should be able to pass the after key with a formatted and a non-formatted date (a string or a long) when format is used. If you have a recreation that doesn't work please open a new issue.