400 in the elasticsearch search request with Buildhub stage

[Natim]

curl 'https://buildhub.stage.mozaws.net/v1/buckets/buildhub/collections/releases/search' --data '{"aggs":{"target.version3":{"filter":{},"aggs":{"target.version":{"terms":{"field":"target.version","size":1000,"order":{"_term":"desc"}}},"target.version_count":{"cardinality":{"field":"target.version"}}}},"target.platform4":{"filter":{},"aggs":{"target.platform":{"terms":{"field":"target.platform","size":1000,"order":{"_term":"asc"}}},"target.platform_count":{"cardinality":{"field":"target.platform"}}}},"target.channel5":{"filter":{},"aggs":{"target.channel":{"terms":{"field":"target.channel","size":1000,"order":{"_term":"asc"}}},"target.channel_count":{"cardinality":{"field":"target.channel"}}}},"target.locale6":{"filter":{},"aggs":{"target.locale":{"terms":{"field":"target.locale","size":1000,"order":{"_term":"asc"}}},"target.locale_count":{"cardinality":{"field":"target.locale"}}}},"source.product9":{"filter":{},"aggs":{"source.product":{"terms":{"field":"source.product","size":50}},"source.product_count":{"cardinality":{"field":"source.product"}}}}},"size":30,"sort":[{"download.date":"desc"}],"highlight":{"fields":{"source.product":{},"target.channel":{},"target.version":{},"target.locale":{},"target.platform":{},"build.id":{}}}}'

[Natim]

{"error":"Bad Request","errno":107,"code":400,"details":{"reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [target.version] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","type":"illegal_argument_exception"},"message":"all shards failed"}

[leplatrem]

Do you know which version of ES is running ?

It should be 5,X

Amazon already runs it...
https://aws.amazon.com/about-aws/whats-new/2017/01/elasticsearch-5-now-available-on-amazon-elasticsearch-service/

[leplatrem]

I believe that this is due to the fact that our mapping was not loaded, because the collection metadata are empty: https://buildhub.stage.mozaws.net/v1/buckets/buildhub/collections/releases

[Natim]

@leplatrem can we create it now or is it too late?

[leplatrem]

If I'm not wrong, the index has to be dropped and re-recreated. In this case, existing records will have to be re-indexed... See Kinto/kinto-elasticsearch#20

[leplatrem]

Now that #231 was implemented, it shouldn't happen with initial loading. It doesn't seem we're safe for major index schema changes in the future

[leplatrem]

With #242 we'll have another safety belt to avoid that