ElasticsearchRestRemplate bulk delete by ID removes all objects from index [DATAES-525] #1097
Description
crombach opened DATAES-525 and commented
I am using an ElasticsearchRestTemplate instance as the implementation of the ElasticsearchOperations bean for my Spring application because I need to interface with an AWS Elasticsearch cluster.
When I try to perform a bulk delete operation via ElasticsearchRestTemplate#delete(DeleteQuery deleteQuery, Class<T> clazz), the entire contents of my index are deleted no matter what DeleteQuery I send.
This example query shows essentially what I am trying to do, but with my own domain classes and different IDs.
DeleteQuery query = new DeleteQuery();
query.setQuery(QueryBuilders.idsQuery().addIds("1", "2", "3");
elasticsearchOperations.delete(query, DomainObject.class);I believe the reason all the entities in the index are deleted is because the QueryBuilder set on the DeleteQuery is not honored.
ElasticsearchRestTemplate#delete(DeleteQuery deleteQuery, Class<T> clazz) contains this code snippet, which looks perfectly reasonable:
SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(deleteQuery.getQuery()).withIndices(indexName)
.withTypes(typeName).withPageable(PageRequest.of(0, pageSize)).build();
...
Page<String> scrolledResult = startScroll(scrollTimeInMillis, searchQuery, String.class, onlyIdResultMapper);However, the version of the startScroll method used above calls doScroll(SearchRequest request, SearchQuery searchQuery), which does not honor the passed SearchQuery's query. Instead, it only uses the SearchQuery's filter, which cannot be set by the caller of ElasticsearchRestTemplate#delete(DeleteQuery deleteQuery, Class<T> clazz):
private SearchResponse doScroll(SearchRequest request, SearchQuery searchQuery) {
Assert.notNull(searchQuery.getIndices(), "No index defined for Query");
Assert.notNull(searchQuery.getTypes(), "No type define for Query");
Assert.notNull(searchQuery.getPageable(), "Query.pageable is required for scan & scroll");
if (searchQuery.getFilter() != null) {
request.source().postFilter(searchQuery.getFilter());
}
request.source().version(true);
try {
return client.search(request);
} catch (IOException e) {
throw new ElasticsearchException("Error for search request with scroll: " + request.toString(), e);
}
}This results in all entities in the index being returned by the scroll operation, and thus all the entities are deleted. It seems to me that doScroll should also have a block like this before the first "if" statement:
if (searchQuery.getQuery() != null) {
request.source().query(searchQuery.getQuery());
}This way, only the entities that match the query will be returned from the scroll and thus only the expected entities will be deleted
Affects: 3.2 M1 (Moore)
Issue Links:
- DATAES-522 ElasticsearchRestTemplate.delete(),add withFilter()
("supersedes")