Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use ElasticSearch Java Client API #987

Closed
reckart opened this issue Feb 27, 2019 · 0 comments · Fixed by #1181
Closed

Use ElasticSearch Java Client API #987

reckart opened this issue Feb 27, 2019 · 0 comments · Fixed by #1181
Assignees
@az79nefy
Labels
Module: ElasticSearch Module: External Search ⚙️ Refactoring
Milestone
0.10.0

Comments

@reckart
Copy link
Member

reckart commented Feb 27, 2019

Describe the refactoring action
We are currently using hand-crafted Java object and at times hand-crafted JSON objects to communicate with ElasticSearch servers. This makes our code unnecessarily complex. The benefit is basically saving some dependencies on ElasticSearch and thus potentially avoiding version conflicts e.g. on Lucene. However, we should investigate if we can use the ElasticSearch Java Client API without too much of a dependency impact to make our code more maintainable.

See: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-search.html

Expected benefit
Easier to read and more maintainable code, in particular as we make increased use of different ElasticSearch features and would piece-by-piece end up re-implementing the ElasticSearch Client API in our own codebase.
@reckart reckart added this to the 0.9.0 milestone Feb 27, 2019
@reckart reckart mentioned this issue Feb 27, 2019
Merged
@UWinch UWinch self-assigned this Mar 1, 2019
@jcklie jcklie mentioned this issue Mar 5, 2019
Closed
4 tasks
@reckart reckart modified the milestones: 0.9.0, 0.10.0 May 7, 2019
@jcklie jcklie assigned az79nefy and unassigned UWinch May 9, 2019
az79nefy added a commit that referenced this issue May 16, 2019
@az79nefy
#987 Use ElasticSearch Java Client API
899d027 
- replaced manual creation of ElasticSearch queries with Java Client API
@az79nefy az79nefy mentioned this issue May 16, 2019
Merged
4 tasks
reckart added a commit that referenced this issue May 17, 2019
@reckart
#987 - Use ElasticSearch Java Client API
d02835a 
- Enforce consistent version of ElasticSearch across modules
reckart added a commit that referenced this issue May 17, 2019
@reckart
#987 - Use ElasticSearch Java Client API
1f2d980 
- Removed unused JSON classes
reckart added a commit that referenced this issue May 24, 2019
@reckart
#987 - Use ElasticSearch Java Client API
7325a87 
- Switch from transport to REST high-level client
- Set ElasticSearch version such that its Lucene dependency aligns with the one we have for MTAS and RDF4J
az79nefy added a commit that referenced this issue May 28, 2019
@az79nefy
#987 Use ElasticSearch Java Client API
d20024f 
- added ability to do random ranking of query results
- introduced constant for strings such as "metadata"
az79nefy added a commit that referenced this issue May 28, 2019
@az79nefy
#987 Use ElasticSearch Java Client API
9e6b196 
- added querying of individual documents (GetRequest) for Java API
- removed more unused JSON classes
- added test to assert that retrieved document text is not null
az79nefy added a commit that referenced this issue May 28, 2019
@az79nefy
#987 Use ElasticSearch Java Client API
5f12401 
- fixed dependencies
az79nefy added a commit that referenced this issue May 28, 2019
@az79nefy
#987 Use ElasticSearch Java Client API
ff28f09 
- the key of the highlights map is the default field
- modified documentation about the expected fields of the ES document
@qpwo qpwo mentioned this issue May 29, 2019
Merged
2 tasks
reckart pushed a commit that referenced this issue May 31, 2019
@qpwo @reckart
#987 - Use ElasticSearch Java Client API (via #1221)
6455b28 
* made random search constantify the match quality so that result orderings are properly random. for issue #16
* added random seeding (need to change fixed seed)
* switch to aTraits.getSeed(), which still needs to somehow be different for different users
reckart added a commit that referenced this issue Jun 4, 2019
@reckart
Merge pull request #1181 from inception-project/feature/987-Use-Elast…
33723a3 
…icSearch-Java-Client-API

#987 Use ElasticSearch Java Client API
reckart added a commit that referenced this issue Jun 4, 2019
@reckart
Merge branch 'master' into feature/1227-integrate-and-clean-up-pdfext…
e679a8e 
…ract-code

* master:
  #987 - Use ElasticSearch Java Client API (via #1221)
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
reckart added a commit that referenced this issue Jun 4, 2019
@reckart
Merge branch 'master' into feature/1132-unify-eval-train-data
6e3ab40 
* master:
  #1215  Write recommender evaluation scores to event log
  #987 - Use ElasticSearch Java Client API (via #1221)
  #1215 Write recommender evaluation scores to event log
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 - Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
reckart added a commit that referenced this issue Jun 5, 2019
@reckart
Merge branch 'master' into feature/867-group_search_results_by_featur…
64df0c3 
…e_value

* master: (37 commits)
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #987 - Use ElasticSearch Java Client API (via #1221)
  #804 easy way to resume annotation
  #804 - Easy way to resume annotation
  #1132 Unify training data
  #1132 Unify training data
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #1132 Unify training data
  #1132 Unify training data
  #1132 Unify training data -bug fix
  #1132 Unify training data selection for evaluation - cannot use evalsim specific labeling of x-axis because of annopage chart - fit x-axis to data points
  #987 - Use ElasticSearch Java Client API
  #1132 Unify training data selection for evaluation
  #804 Easy way to resume annotation
  ...
reckart added a commit that referenced this issue Jun 6, 2019
@reckart
Merge branch 'master' into bugfix/1235-Bulk-operations-are-very-slow
d27bd85 
* master: (48 commits)
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #1227 - Integrate and clean up PDFExtract code
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #987 - Use ElasticSearch Java Client API (via #1221)
  #804 easy way to resume annotation
  #804 - Easy way to resume annotation
  #1132 Unify training data
  #1132 Unify training data
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  #987 Use ElasticSearch Java Client API
  ...

% Conflicts:
%	inception-ui-search/src/main/java/de/tudarmstadt/ukp/inception/app/ui/search/sidebar/SearchAnnotationSidebar.java
reckart added a commit that referenced this issue Jun 6, 2019
@reckart
Merge branch 'master' into 0.10.x-uimav3
e7f4826 
* master: (60 commits)
  #1194 - Predict annotations directly instead of using PredictedSpan
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #1227 - Integrate and clean up PDFExtract code
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #867 - Group search results by feature value
  #1194 - Predict annotations directly instead of using PredictedSpan
  #1215  Write recommender evaluation scores to event log
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #1227 - Integrate and clean up PDFExtract code
  #987 - Use ElasticSearch Java Client API (via #1221)
  #1215 Write recommender evaluation scores to event log
  #804 easy way to resume annotation
  #804 - Easy way to resume annotation
  #1132 Unify training data
  #1132 Unify training data
  ...

% Conflicts:
%	inception-app-webapp/pom.xml
%	inception-concept-linking/src/main/java/de/tudarmstadt/ukp/inception/conceptlinking/recommender/NamedEntityLinker.java
%	inception-example-imls-data-majority/pom.xml
%	inception-imls-external/src/test/java/de/tudarmstadt/ukp/inception/recommendation/imls/external/ExternalRecommenderIntegrationTest.java
%	inception-recommendation/src/test/java/de/tudarmstadt/ukp/inception/recommendation/service/RecommendationServiceImplIntegrationTest.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@az79nefy az79nefy

Labels
Module: ElasticSearch Module: External Search ⚙️ Refactoring
Projects
None yet
Milestone
0.10.0
Development

Successfully merging a pull request may close this issue.

#987 Use ElasticSearch Java Client API inception-project/inception
3 participants
@reckart @az79nefy @UWinch