spring-projects · sothawo · Aug 18, 2020 · Aug 4, 2020
diff --git a/src/main/asciidoc/preface.adoc b/src/main/asciidoc/preface.adoc
@@ -1,7 +1,8 @@
 [[preface]]
 = Preface
 
-The Spring Data Elasticsearch project applies core Spring concepts to the development of solutions using the Elasticsearch Search Engine. It provides:
+The Spring Data Elasticsearch project applies core Spring concepts to the development of solutions using the Elasticsearch Search Engine.
+It provides:
 
 * _Templates_ as a high-level abstraction for storing, searching, sorting documents and building aggregations.
 * _Repositories_ which for example enable the user to express queries by defining interfaces having customized method names (for basic information about repositories see <<repositories>>).
@@ -29,12 +30,13 @@ Requires an installation of https://www.elastic.co/products/elasticsearch[Elasti
 === Versions
 
 The following table shows the Elasticsearch versions that are used by Spring Data release trains and version of Spring Data Elasticsearch included in that, as well as the Spring Boot versions referring to that particular Spring Data release train:
+
 [cols="^,^,^,^",options="header"]
 |===
 | Spring Data Release Train |Spring Data Elasticsearch |Elasticsearch | Spring Boot
 | 2020.0.0footnote:cdv[Currently in development] |4.1.xfootnote:cdv[]|7.8.1 |2.3.xfootnote:cdv[]
 | Neumann | 4.0.x | 7.6.2 |2.3.x
-| Moore | 3.2.x |6.8.12 | 2.2.x
+| Moore | 3.2.x |6.8.10 | 2.2.x
 | Lovelace | 3.1.x | 6.2.2 |2.1.x
 | Kayfootnote:oom[Out of maintenance] | 3.0.xfootnote:oom[] | 5.5.0 | 2.0.xfootnote:oom[]
 | Ingallsfootnote:oom[] | 2.1.xfootnote:oom[] | 2.4.0 | 1.5.xfootnote:oom[]

diff --git a/src/main/asciidoc/reference/elasticsearch-operations.adoc b/src/main/asciidoc/reference/elasticsearch-operations.adoc
@@ -45,7 +45,8 @@ public class TransportClientConfig extends ElasticsearchConfigurationSupport {
   }
 }
 ----
-<1> Setting up the <<elasticsearch.clients.transport>>. Deprecated as of version 4.0.
+<1> Setting up the <<elasticsearch.clients.transport>>.
+Deprecated as of version 4.0.
 <2> Creating the `ElasticsearchTemplate` bean, offering both names, _elasticsearchOperations_ and _elasticsearchTemplate_.
 ====
 
@@ -75,7 +76,9 @@ public class RestClientConfig extends AbstractElasticsearchConfiguration {
 [[elasticsearch.operations.usage]]
 == Usage examples
 
-As both `ElasticsearchTemplate` and `ElasticsearchRestTemplate` implement the `ElasticsearchOperations` interface, the code to use them is not different. The example shows how to use an injected `ElasticsearchOperations` instance in a Spring REST controller. The decision, if this is using the `TransportClient` or the `RestClient` is made by providing the corresponding Bean with one of the configurations shown above.
+As both `ElasticsearchTemplate` and `ElasticsearchRestTemplate` implement the `ElasticsearchOperations` interface, the code to use them is not different.
+The example shows how to use an injected `ElasticsearchOperations` instance in a Spring REST controller.
+The decision, if this is using the `TransportClient` or the `RestClient` is made by providing the corresponding Bean with one of the configurations shown above.
 
 .ElasticsearchOperations usage
 ====
@@ -123,9 +126,12 @@ include::reactive-elasticsearch-operations.adoc[leveloffset=+1]
 [[elasticsearch.operations.searchresulttypes]]
 == Search Result Types
 
-When a document is retrieved with the methods of the  `DocumentOperations` interface, just the found entity will be returned. When searching with the methods of the `SearchOperations` interface, additional information is available for each entity, for example the _score_ or the _sortValues_ of the found entity.
+When a document is retrieved with the methods of the  `DocumentOperations` interface, just the found entity will be returned.
+When searching with the methods of the `SearchOperations` interface, additional information is available for each entity, for example the _score_ or the _sortValues_ of the found entity.
 
-In order to return this information, each entity is wrapped in a `SearchHit` object that contains this entity-specific additional information. These `SearchHit` objects themselves are returned within a `SearchHits` object which additionally contains informations about the whole search like the _maxScore_ or requested aggregations. The following classes and interfaces are now available:
+In order to return this information, each entity is wrapped in a `SearchHit` object that contains this entity-specific additional information.
+These `SearchHit` objects themselves are returned within a `SearchHits` object which additionally contains informations about the whole search like the _maxScore_ or requested aggregations.
+The following classes and interfaces are now available:
 
 .SearchHit<T>
 Contains the following information:
@@ -155,3 +161,108 @@ Returned by the low level scroll API functions in `ElasticsearchRestTemplate`, i
 .SearchHitsIterator<T>
 An Iterator returned by the streaming functions of the `SearchOperations` interface.
 
+== Queries
+
+Almost all of the methods defined in the `SearchOperations` and `ReactiveSearchOperations` interface take a `Query` parameter that defines the query to execute for searching. `Query` is an interface and Spring Data Elasticsearch provides three implementations: `CriteriaQuery`, `StringQuery` and `NativeSearchQuery`.
+
+=== CriteriaQuery
+
+`CriteriaQuery` based queries allow the creation of queries to search for data without knowing the syntax or basics of Elasticsearch queries. They allow the user to build queries by simply chaining and combining `Criteria` objects that specifiy the criteria the searched documents must fulfill.
+
+NOTE: when talking about AND or OR when combining criteria keep in mind, that in Elasticsearch AND are converted to a **must** condition and OR to a **should**
+
+`Criteria` and their usage are best explained by example
+(let's assume we have a `Book` entity with a `price` property):
+
+.Get books with a given price
+====
+[source,java]
+----
+Criteria criteria = new Criteria("price").is(42.0);
+Query query = new CriteriaQuery(criteria);
+----
+====
+
+Conditions for the same field can be chained, they will be combined with a logical AND:
+
+.Get books with a given price
+====
+[source,java]
+----
+Criteria criteria = new Criteria("price").greaterThan(42.0).lessThan(34.0L);
+Query query = new CriteriaQuery(criteria);
+----
+====
+
+When chaining `Criteria`, by default a AND logic is used:
+
+.Get all persons with first name _James_  and last name _Miller_:
+====
+[source,java]
+----
+Criteria criteria = new Criteria("lastname").is("Miller") <1>
+  .and("firstname").is("James")                           <2>
+Query query = new CriteriaQuery(criteria);
+----
+<1> the first `Criteria`
+<2> the and() creates a new `Criteria` and chaines it to the first one.
+====
+
+If you want to create nested queries, you need to use subqueries for this. Let's assume we want to find all persons with a last name of _Miller_ and a first name of either _Jack_ or _John_:
+
+.Nested subqueries
+====
+[source,java]
+----
+Criteria miller = new Criteria("lastName").is("Miller")  <.>
+  .subCriteria(                                          <.>
+    new Criteria().or("firstName").is("John")            <.>
+      .or("firstName").is("Jack")                        <.>
+  );
+Query query = new CriteriaQuery(criteria);
+----
+<.> create a first `Criteria` for the last name
+<.> this is combined with AND to a subCriteria
+<.> This sub Criteria is an OR combination for the first name _John_
+<.> and the first name Jack
+====
+
+Please refer to the API documentation of the `Criteria` class for a complete overview of the different available operations.
+
+=== StringQuery
+
+This class takes an Elasticsearch query as JSON String.
+The following code shows a query that searches for persons having the first name "Jack":
+
+====
+[source,java]
+----
+
+Query query = new SearchQuery("{ \"match\": { \"firstname\": { \"query\": \"Jack\" } } } ");
+SearchHits<Person> searchHits = operations.search(query, Person.class);
+
+----
+====
+
+Using `StringQuery` may be appropriate if you already have an Elasticsearch query to use.
+
+=== NativeSearchQuery
+
+`NativeSearchQuery` is the class to use when you have a complex query, or a query that cannot be expressed by using the `Criteria` API, for example when building queries and using aggregates.
+It allows to use all the different `QueryBuilder` implementations from the Elasticsearch library therefore named "native".
+
+The following code shows how to search for persons with a given firstname and for the found documents have a terms aggregation that counts the number of occurences of the lastnames for these persons:
+
+====
+[source,java]
+----
+Query query = new NativeSearchQueryBuilder()
+    .addAggregation(terms("lastnames").field("lastname").size(10)) //
+    .withQuery(QueryBuilders.matchQuery("firstname", firstName))
+    .build();
+
+SearchHits<Person> searchHits = operations.search(query, Person.class);
+----
+====
+
+
diff --git a/src/main/java/org/springframework/data/elasticsearch/core/CriteriaFilterProcessor.java b/src/main/java/org/springframework/data/elasticsearch/core/CriteriaFilterProcessor.java
@@ -17,9 +17,11 @@
 
 import static org.springframework.data.elasticsearch.core.query.Criteria.*;
 
+import java.util.ArrayList;
 import java.util.Iterator;
 import java.util.LinkedList;
 import java.util.List;
+import java.util.stream.Collectors;
 
 import org.elasticsearch.common.geo.GeoDistance;
 import org.elasticsearch.index.query.BoolQueryBuilder;
@@ -47,66 +49,56 @@
  */
 class CriteriaFilterProcessor {
 
-	QueryBuilder createFilterFromCriteria(Criteria criteria) {
-		List<QueryBuilder> fbList = new LinkedList<>();
-		QueryBuilder filter = null;
+	@Nullable
+	QueryBuilder createFilter(Criteria criteria) {
+
+		List<QueryBuilder> filterBuilders = new ArrayList<>();
 
 		for (Criteria chainedCriteria : criteria.getCriteriaChain()) {
-			QueryBuilder fb = null;
+
 			if (chainedCriteria.isOr()) {
-				fb = QueryBuilders.boolQuery();
-				for (QueryBuilder f : createFilterFragmentForCriteria(chainedCriteria)) {
-					((BoolQueryBuilder) fb).should(f);
-				}
-				fbList.add(fb);
+				BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
+				queriesForEntries(chainedCriteria).forEach(boolQuery::should);
+				filterBuilders.add(boolQuery);
 			} else if (chainedCriteria.isNegating()) {
 				List<QueryBuilder> negationFilters = buildNegationFilter(criteria.getField().getName(),
 						criteria.getFilterCriteriaEntries().iterator());
 
-				if (!negationFilters.isEmpty()) {
-					fbList.addAll(negationFilters);
-				}
+				filterBuilders.addAll(negationFilters);
 			} else {
-				fbList.addAll(createFilterFragmentForCriteria(chainedCriteria));
+				filterBuilders.addAll(queriesForEntries(chainedCriteria));
 			}
 		}
 
-		if (!fbList.isEmpty()) {
-			if (fbList.size() == 1) {
-				filter = fbList.get(0);
+		QueryBuilder filter = null;
+
+		if (!filterBuilders.isEmpty()) {
+
+			if (filterBuilders.size() == 1) {
+				filter = filterBuilders.get(0);
 			} else {
-				filter = QueryBuilders.boolQuery();
-				for (QueryBuilder f : fbList) {
-					((BoolQueryBuilder) filter).must(f);
-				}
+				BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
+				filterBuilders.forEach(boolQuery::must);
+				filter = boolQuery;
 			}
 		}
+
 		return filter;
 	}
 
-	private List<QueryBuilder> createFilterFragmentForCriteria(Criteria chainedCriteria) {
-		Iterator<Criteria.CriteriaEntry> it = chainedCriteria.getFilterCriteriaEntries().iterator();
-		List<QueryBuilder> filterList = new LinkedList<>();
+	private List<QueryBuilder> queriesForEntries(Criteria criteria) {
 
-		String fieldName = chainedCriteria.getField().getName();
+		Assert.notNull(criteria.getField(), "criteria must have a field");
+		String fieldName = criteria.getField().getName();
 		Assert.notNull(fieldName, "Unknown field");
-		QueryBuilder filter = null;
 
-		while (it.hasNext()) {
-			Criteria.CriteriaEntry entry = it.next();
-			filter = processCriteriaEntry(entry.getKey(), entry.getValue(), fieldName);
-			filterList.add(filter);
-		}
-
-		return filterList;
+		return criteria.getFilterCriteriaEntries().stream()
+				.map(entry -> queryFor(entry.getKey(), entry.getValue(), fieldName)).collect(Collectors.toList());
 	}
 
 	@Nullable
-	private QueryBuilder processCriteriaEntry(OperationKey key, Object value, String fieldName) {
+	private QueryBuilder queryFor(OperationKey key, Object value, String fieldName) {
 
-		if (value == null) {
-			return null;
-		}
 		QueryBuilder filter = null;
 
 		switch (key) {
@@ -169,8 +161,7 @@ private QueryBuilder processCriteriaEntry(OperationKey key, Object value, String
 					// 2x text
 					twoParameterBBox((GeoBoundingBoxQueryBuilder) filter, valArray);
 				} else {
-					// error
-					Assert.isTrue(false,
+					throw new IllegalArgumentException(
 							"Geo distance filter takes a 1-elements array(GeoBox) or 2-elements array(GeoPoints or Strings(format lat,lon or geohash)).");
 				}
 				break;
@@ -208,8 +199,7 @@ private void oneParameterBBox(GeoBoundingBoxQueryBuilder filter, Object value) {
 
 		GeoBox geoBBox;
 		if (value instanceof Box) {
-			Box sdbox = (Box) value;
-			geoBBox = GeoBox.fromBox(sdbox);
+			geoBBox = GeoBox.fromBox((Box) value);
 		} else {
 			geoBBox = (GeoBox) value;
 		}
@@ -218,7 +208,7 @@ private void oneParameterBBox(GeoBoundingBoxQueryBuilder filter, Object value) {
 				geoBBox.getBottomRight().getLon());
 	}
 
-	private static boolean isType(Object[] array, Class clazz) {
+	private static boolean isType(Object[] array, Class<?> clazz) {
 		for (Object o : array) {
 			if (!clazz.isInstance(o)) {
 				return false;
@@ -247,7 +237,7 @@ private List<QueryBuilder> buildNegationFilter(String fieldName, Iterator<Criter
 		while (it.hasNext()) {
 			Criteria.CriteriaEntry criteriaEntry = it.next();
 			QueryBuilder notFilter = QueryBuilders.boolQuery()
-					.mustNot(processCriteriaEntry(criteriaEntry.getKey(), criteriaEntry.getValue(), fieldName));
+					.mustNot(queryFor(criteriaEntry.getKey(), criteriaEntry.getValue(), fieldName));
 			notFilterList.add(notFilter);
 		}