Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 #177

arjansh · 2018-01-24T10:40:53Z

Biggest changes:

Node client is no longer supported. If you want to connect as a node client, you can "Start locally a Coordinating Only Node and then simply create a TransportClient in your application which connects to this Coordinating Only Node."
Elasticsearch no longer supports the removal of Mappings from indices and existing type and field mappings can no longer be updated. As a result of this, we can no longer support drop table functionality in MetaModel.
The io.searchbox jest client has been replaced by Elasticsearch's High level rest client. As a result of which the complete module has been rewritten and some code could be moved to the common module, because it now is the same in both the native and rest client modules.
The EmbeddedElasticSearchServer has been removed from the test classes, because it is no longer possible to spawn a node in that manner. For the native client, it has been replaced by an Elasticsearch provided test server and client by extending the ESSingleNodeTestCase class, within the Rest client we now spawn a Docker image which runs an Elasticsearch cluster.

…ture/update-elastic-search-to-5.6.3

…ation test)

… integration test runs with docker image.

… clients to common module and reinstated bulk requests for rest client.

…factored touched code based on findings.

… (I presume) date format.

…anges.

…re unneeded code.

kaspersorensen · 2018-01-25T04:47:02Z

You may want to update the title of the PR to be "Upgrade ElasticSearch ..." instead of "Upgrade MetaModel ..." :-) It's a pretty big diff, so I'm gonna check it out locally and provide some feedback shortly.

kaspersorensen

I generally like what I'm seeing here. It even makes me think that we should maybe just drop support entirely for the "native" module (now effectively the "transport protocol" module) entirely. But I guess that there's still a bit of value in keeping that.

kaspersorensen · 2018-01-25T05:08:25Z

...ommon/src/main/java/org/apache/metamodel/elasticsearch/AbstractElasticSearchDataContext.java

+
+    public static final TimeValue TIMEOUT_SCROLL = TimeValue.timeValueSeconds(60);
+
+    protected final Object elasticSearchClient;


There doesn't seem to be any reason to keep this in the abstract class any more, since the Object type removes any kind of possibility of reusing it. It's used differently in the two subtypes, so I suggest they each just have a field for the client object with a type that they need.

kaspersorensen · 2018-01-25T05:12:19Z

elasticsearch/rest/pom.xml

+												<ES_JAVA_OPTS>-Xms1g -Xmx1g</ES_JAVA_OPTS>
+												<cluster.name>docker-cluster</cluster.name>
+												<bootstrap.memory_lock>true</bootstrap.memory_lock>
+											    <xpack.security.enabled>false</xpack.security.enabled>


indentation issues here.

kaspersorensen · 2018-01-25T05:13:44Z

elasticsearch/rest/pom.xml

+					<plugin>
+						<groupId>io.fabric8</groupId>
+						<artifactId>docker-maven-plugin</artifactId>
+						<version>0.15.16</version>


It's a fairly old version of this plugin that you're using here. Maybe we should (1) use the latest just because Docker things like this seems to move fast and (2) add it's version to in the root pom.

arjansh · 2018-01-25T15:55:31Z

I suggest we do a release candidate after this PR has been merged.

ClaudiaPHI and others added 30 commits December 1, 2017 21:07

fixed compilation issues related to upgrade to 5.6.3

36649c8

added DockerFile, improved the deletion query and insertion queries

d5e5bb3

improved the mapping of the created table

a6e3d05

added an UpdateBuilder

5ee7f8f

added improvements to the builders

6d91970

created a docker elasticsearch image to run the tests

c9d7622

get the docker host name from environmental variable

ff966b3

deleted the embedded server as it is not longer used in testing

1569e93

improvements made to the tests

f7f557f

Merge branch 'master' of https://github.com/apache/metamodel into fea…

00ceb5d

…ture/update-elastic-search-to-5.6.3

Bugfixes and changes to the integration test (and setup of the integr…

9f3cc1d

…ation test)

Refactored ElasticSearchDataContext tests.

a1fd545

Refactored code a bit based on sonarQube feedback.

24fbe6b

Intermediate refactoring for Elasticsearch Rest client

8d1acab

Improved working of code and fixed integration test for Rest client.

894e76c

Removed Integration test from native client and make sure rest client…

8bfb7ef

… integration test runs with docker image.

Removed unused import.

72f45c5

Activate integration-test profile and docker service for Travis.

7bb5555

Added some finals, moved some carbon copied code from navtie and rest…

fa491da

… clients to common module and reinstated bulk requests for rest client.

Added integration test for ElasticSearchRestDataContextFactory and re…

67f4f8a

…factored touched code based on findings.

Reinstated "scrolling" on rest client.

0aacbb9

Reordered code a bit too avoid copy/paste code.

4e1db99

Did some sonar inspired improvements to the code.

f94b260

Extended Date Converter so it is able to handle the new Elasticsearch…

6d843e8

… (I presume) date format.

Improved code a bit.

2715857

Reverted some out of context changes.

8e1c121

Reverted some more out of context code (style) changes.

9002155

And some more unnecessary out of context changes reverted.

b2b8493

Reverted some more out of context and unwanted tabs to spaces changes.

3c23da2

Even more reverting of unneeded out of context (mainly code style) ch…

9ca4a22

…anges.

arjansh added 2 commits January 24, 2018 09:27

Stabalized testCreateContextAndBulkScript test.

9f799bd

Fixed indentation (tabs to spaces and vice versa) and removed some mo…

bf955f5

…re unneeded code.

kaspersorensen reviewed Jan 25, 2018

View reviewed changes

arjansh changed the title ~~Upgrade MetaModel dependency from 1.5.0 to 5.6.3~~ Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 Jan 25, 2018

Addressed feedback on Pull request.

dbdbb10

asfgit closed this in bda8d76 Jan 26, 2018

arjansh deleted the feature/update-elastic-search-to-5.6.3 branch June 21, 2019 10:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 #177

Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 #177

arjansh commented Jan 24, 2018

kaspersorensen commented Jan 25, 2018

kaspersorensen left a comment

kaspersorensen Jan 25, 2018

kaspersorensen Jan 25, 2018

kaspersorensen Jan 25, 2018

arjansh commented Jan 25, 2018


		public static final TimeValue TIMEOUT_SCROLL = TimeValue.timeValueSeconds(60);

		protected final Object elasticSearchClient;

Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 #177

Upgrade Elasticsearch dependency from 1.5.0 to 5.6.3 #177

Conversation

arjansh commented Jan 24, 2018

kaspersorensen commented Jan 25, 2018

kaspersorensen left a comment

Choose a reason for hiding this comment

kaspersorensen Jan 25, 2018

Choose a reason for hiding this comment

kaspersorensen Jan 25, 2018

Choose a reason for hiding this comment

kaspersorensen Jan 25, 2018

Choose a reason for hiding this comment

arjansh commented Jan 25, 2018