Moving from elasticsearch to opendistro for elasticsearch

[HaZet1968]

Please describe a way how to move from elasticsearch (default download) to opendistro for elasticsearch. Really interested in it.

[Jeeppler]

I also want to use Open Distro for Elasticsearch rather than the available version on Elasticsearch service. The alerting and SQL features are super attractive.

[aetter]

Hi @HaZet1968, great suggestion. For indices, you can use snapshots to migrate between clusters. I don't know enough about the default download's alerting feature to say whether you can somehow migrate your alerts, but I suspect you'd have to recreate them by hand in Open Distro for Elasticsearch.

We currently have no documentation on snapshot and restore, but it's in my backlog. If anyone wants to tackle it, I think docs/elasticsearch/snapshot-restore.md would be the appropriate place.

[Jeeppler]

@aetter My question is not so much about moving an existing cluster. I am interested on running OpenDistro for Elasticsearch in AWS. How and when will OpenDistro for ElasticSearch available on AWS ElasticSearch service and how will we be able to migrate to it or will ElasticSearch version 6.5 on ElasticSearch service automatically be based on OpenDistro?

[aetter]

Hi @Jeeppler, I don't have anything to share on when. Regarding features, I think the announcement blog post provides the best summary. "With the first release, our goal is to address many critical features missing from open source Elasticsearch, such as security, event monitoring and alerting, and SQL support. Many of these features are ones that we have been working on for inclusion in Amazon Elasticsearch Service."

If you were to configure an Open Distro for Elasticsearch cluster today on your own hardware, EC2, etc. and then want to migrate to Amazon Elasticsearch Service at a later date, you'd probably perform the migration using a snapshot.

Alerting, for example, stores its information in several indices, which would be included in the snapshot. If/when the feature comes to Amazon Elasticsearch Service, I suspect monitors will migrate pretty cleanly.

The Security plugin, on the other hand, makes heavy use of YML files, which you don't have direct access to on Amazon Elasticsearch Service. If/when the feature comes to Amazon ES, I suspect that migrating a complex security configuration will be a more manual process.

Open Distro for Elasticsearch gives us a chance to "race ahead" of the service, contribute back to an ecosystem that we love, and work with the community to see what works and what doesn't.

[Jeeppler]

@aetter thanks for clarifying the situation. When I read the post from Jeff Barr I was actually under the impression the features are available on AWS Elasticsearch Service today, mostly because of this sentence:

"Let’s take a quick look at the features that we are including in Open Distro for Elasticsearch. Some of these are currently available in Amazon Elasticsearch Service; others will become available in future updates." (source: https://aws.amazon.com/blogs/aws/new-open-distro-for-elasticsearch/)

My assumption was based on the experience that Jeff Barr mostly announces new features and services in AWS.

However, one more thing which is not clear from the source you cited: "With the first release". I assume the first release refers to Open Distro for Elasticsearch version 0.7. Is my interpretation accurate?

[aetter]

@Jeeppler, yep, "with the first release" means 0.7.0. Sorry about the confusion from the blog post; Amazon ES currently has none of the Open Distro for Elasticsearch plugins (Alerting, Security, SQL, or PA).

[Jeeppler]

@aetter thanks again for the clarification.

[aetter]

Closing in favor of #27.