0.40.2 still not compatible with elasticsearch-cloud-aws plugin

[jeffnappi]

This plugin is extremely useful and would be great for discovery and cluster allocation awareness.

root@node:/usr/share/crate# bin/plugin install elasticsearch/elasticsearch-cloud-aws/2.2.0
-> Installing elasticsearch/elasticsearch-cloud-aws/2.2.0...
Trying http://download.elasticsearch.org/elasticsearch/elasticsearch-cloud-aws/elasticsearch-cloud-aws-2.2.0.zip...
Downloading ... DONE
Installed elasticsearch/elasticsearch-cloud-aws/2.2.0 into /usr/share/crate/plugins/cloud-aws
root@node:/usr/share/crate# bin/crate
[2014-07-27 04:45:57,912][INFO ][node                     ] [Mar-Vell] version[1.2.2], pid[4026], build[${build/NA]
[2014-07-27 04:45:57,913][INFO ][node                     ] [Mar-Vell] initializing ...
[2014-07-27 04:45:57,996][INFO ][plugins                  ] [Mar-Vell] loaded [blob, sql, udc, crate-core, cloud-aws, admin-ui], sites [bigdesk, head, crate-admin, segmentspy]
{1.2.2}: Initialization Failed ...
- NoClassDefFoundError[org/elasticsearch/common/collect/Lists]
        ClassNotFoundException[org.elasticsearch.common.collect.Lists]

cloud-aws 2.2.0 should be compatible w/ ES 1.2.2 used in Crate 0.40.2? https://github.com/elasticsearch/elasticsearch-cloud-aws/tree/es-1.2

[mfelsche]

Te reason for this issue is still the same as it was reported for crate 0.28.X: http://stackoverflow.com/questions/22321309/crate-0-28-0-1-and-elasticsearch-cloud-aws-plugin-compatibility

Elasticsearch makes extensive use of shading (see http://maven.apache.org/plugins/maven-shade-plugin/ and https://github.com/elasticsearch/elasticsearch/blob/v1.3.0/pom.xml#L665) That is, bundling dependencies under another namespace (org.elasticsearch.common) in their own jar. The elasticsearch-cloud-aws plugin has been programmed against the elasticsearch jar. We developed crate against the elasticsearch sources as we had to dig a little deeper and we needed to get rid of the shaded dependencies. Long story short, (vanilla) elasticsearch plugins programmed against the elasticsearch jar won't work with crate due to the shading.

The referenced org.elasticsearch.common.collect.Lists comes from googles guava.
One would have to fork the plugin and refactor all the dependencies to org.elasticsearch.common to point to their original sources.

It is a pitty, and I'm really sorry for that, but we currently don't have the time to dig into all the possibly useful elasticsearch plugins to make them usable with crate. But, naturally, we would appreciate any engagement.

[jeffnappi]

I understand. Thank you for the explanation. Our use-case is primarily for availability zone awareness. And we've been able to accomplish that by simply using the AWS api in our startup scripts.

[mfelsche]

Oh, good to hear that you found a workaround! :)
We will keep this issue in mind. Meanwhile, I will close this issue.
Thanks! :)

[pires]

@mfelsche has this changed since then? Really looking forward to add Kubernetes discovery support.

[mfelsche]

Unfortunately i have to say: not yet!
But we still a solution for this on our backlog.
So stay tuned! :)