Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Upgrade to Elasticsearch 5.3 #451
The API in Elasticsearch 5.x has changed quite a bit compared to 2.x. This PR ports the code to 5.3, the functionalities are roughly the same as previously except:
Note: in order to use this PR in distributed mode, you'll have to use Storm 1.1.0 (see #450 ) or upgrade its logging dependencies by hand. This should not be necessary if you run it in local mode.
I will soon publish a comparison of the performance and functionalities of ES5.3 in a blog
BTW am keeping the intermediate commits as some come from master and they contain useful information in isolation.
With this version, we run in the following error:
I found the following difference between elasticsearch 2.4.1 and 5.1.2 :
In 2.4.1. the object in the RangeQuery was treaten as ByteRef after transport.
In 5.1.2. the value here is instanceof Date, the ugly date.toString is used and throws the error :(
As a workaround, I use the following in the ElasticSearchSpout:
and the crawler works fine.
thanks @dstraub, looks like a problem with Elasticsearch but we could take care of generating a clean string and/or specify the format explicitely in the mapping. Since this is used by the AggregationSpout as well it would be worth moving it to the abstract class.
BTW did you change anything to the pom files? I haven't managed to get past the dependency problem with the logging.
I resolved the logging jar hell in my client pom:
Apr 5, 2017