feature: reusable and efficient CtQueries (#1090)

* Support of CtScanner based queries

- Extraction of code to CtBaseQuery to make code cleaner and to allow
reuse of queries
- CtFunction can return Iterator now too

* added concept diagram

* Simplified and thread safe implementation of CtBaseQueryImpl

* fix exception handling. Improve tests

* Remove generic from CtQueryable, add CtQuery#list(Class)

* rename CtBaseQuery#apply to evaluate

Change CtConsumable#accept ... CtConsumer<R> to CtConsumer<Object>

* added sample query: OverridenMethodQuery + tests

* fix compilation error

* fix compilation error

* add missing javadoc and CtQuery#setInput

* fix return type of setInput

* merge CtBaseQueryImpl into CtQueryImpl

* removes Step + doc

* improve query test

* return CtQueryImpl instead of CtQuery

* fix checkstyle problem

* remove by mistake committed test

* pass IntercessionTest

* improve tests and fix NPE

* <R extends Object> List<R> list()

* remove useless generic <T extends CtQuery> T setInput(Object... input);

* remove query-elements.png

* add useless generic to pass the test

* update tests

* update doc

* fix the documentation

* javaDoc JDK 8 compatibility

* improve CtQueryImpl code and fix documentation

doc/filter.md

@@ -39,35 +39,41 @@ list3 = rootPackage.filterChildren(
       return field.getModifiers.contains(ModifierKind.PUBLIC);
     }
   }
-);
+).list();
 ```
 
 Queries
 -------
 
 The Query, introduced in Spoon 5.5 by Pavel Vojtechovsky, is an improved filter mechanism:
 
-* matching can be done with a Java 8 lambda
+* queries can be done with a Java 8 lambda
 * queries can be chained
+* queries can be reused on multiple input elements
 
-`CtQueryable#filterChildren(Filter)` is a filtering query that can be chained:
+**Queries with Java8 lambdas**: `CtQueryable#map(CtFunction)`enables you to give Java 8 lambda as query.
+
+```java
+// returns a list of String
+list = package.map((CtClass c) -> c.getSimpleName()).list();
+```
+
+**Compatibility with existing filters** `CtQueryable#filterChildren(Filter)` is a filtering query that can be chained:
 
 ```java
 // collecting all methods of deprecated classes
 list2 = rootPackage
-    .filterChildren(new AnnotationFilter(Deprecated.class))
-    .filterChildren(new TypeFilter(CtMethod.class)).list();
+    .filterChildren(new AnnotationFilter(Deprecated.class)).list()
 ```
 
-`CtQueryable#map(CtFunction)`enables you to give Java 8 lambda as query.
 A boolean return value of the lambda tells whether the elements are selected for inclusion or not.
 
 ```java
 // creating a custom filter to select all public fields using java 8 lambda
 list3 = rootPackage.filterChildren((CtField field)->field.getModifiers.contains(ModifierKind.PUBLIC)).list();
 ```
 
-If the CtFunction returns an object, this object is given as result to the query:
+**Chaining** If the CtFunction returns an object, this object is given as result to the next step of the query:
 
 ```java
 // a query which processes all not deprecated methods of all deprecated classes
@@ -81,3 +87,14 @@ rootPackage
 
 Finally, if the CtFunction returns and Iterable or an Array then each item of the collection/array is sent to next query step or result.
 
+**Query reuse**. Method `setInput` allows you to reuse the same query over multiple inputs.
+
+```java
+// here is the query
+CtQuery q = new CtQueryImpl().map((CtClass c) -> c.getSimpleName());
+// using it on a first input
+String s1 = q.setInput(cls).list().get(0);
+// using it on a second input
+String s2 = q.setInput(cls2).list().get(0);
+```
+

src/main/java/spoon/reflect/visitor/chain/CtConsumableFunction.java

@@ -0,0 +1,39 @@
+/**
+ * Copyright (C) 2006-2016 INRIA and contributors
+ * Spoon - http://spoon.gforge.inria.fr/
+ *
+ * This software is governed by the CeCILL-C License under French law and
+ * abiding by the rules of distribution of free software. You can use, modify
+ * and/or redistribute the software under the terms of the CeCILL-C license as
+ * circulated by CEA, CNRS and INRIA at http://www.cecill.info.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the CeCILL-C License for more details.
+ *
+ * The fact that you are presently reading this means that you have had
+ * knowledge of the CeCILL-C license and that you accept its terms.
+ */
+package spoon.reflect.visitor.chain;
+
+/**
+ * Represents a function, as {@link CtFunction}. However, the main difference is that
+ * while a {@link CtFunction} returns something with a standard Java return keyword,
+ * a {@link CtConsumableFunction} returns something by passing the returned object
+ * as parameter to the given outpuConsumer#accept. This enables to write efficient and concise code in certain situations.
+ * It also enables one to emulate several returns, by simply calling several times accept, while not paying
+ * the code or performance price of creating a list or an iterable object.
+ *
+ * It is typically used as parameter of {@link CtQueryable#map(CtConsumableFunction)}, can be written as one-liners
+ * with Java8 lambdas:.`cls.map((CtClass&t;?> c, CtConsumer&t;Object> out)->out.accept(c.getParent()))`
+ *
+ * @param <T> the type of the input to the function
+ */
+public interface CtConsumableFunction<T> {
+	/**
+	 * Evaluates the function on the given input.
+	 * @param input the input of the function
+	 * @param outputConsumer the consumer which accepts the results of this function.
+	 */
+	void apply(T input, CtConsumer<Object> outputConsumer);
+}

src/main/java/spoon/reflect/visitor/chain/CtConsumer.java

@@ -0,0 +1,27 @@
+/**
+ * Copyright (C) 2006-2016 INRIA and contributors
+ * Spoon - http://spoon.gforge.inria.fr/
+ *
+ * This software is governed by the CeCILL-C License under French law and
+ * abiding by the rules of distribution of free software. You can use, modify
+ * and/or redistribute the software under the terms of the CeCILL-C license as
+ * circulated by CEA, CNRS and INRIA at http://www.cecill.info.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the CeCILL-C License for more details.
+ *
+ * The fact that you are presently reading this means that you have had
+ * knowledge of the CeCILL-C license and that you accept its terms.
+ */
+package spoon.reflect.visitor.chain;
+
+/**
+ * The functional interface used to receive objects.
+ * It is used for example to receive results of the query in {@link CtQuery#forEach(CtConsumer)}
+ *
+ * @param <T> - the type of accepted elements
+ */
+public interface CtConsumer<T> {
+	void accept(T t);
+}

src/main/java/spoon/reflect/visitor/chain/CtFunction.java

@@ -19,7 +19,7 @@
 /**
  * Abstraction for functions in the Spoon realm.
  * It is used in the query stack, for example by {@link CtQueryable#map(CtFunction)}
- * Compatible with Java 8 lambdas, hence enable to write queries with lambdas.
+ * It is compatible with Java 8 lambdas, hence enable to write one-liner queries with lambdas.
  *
  * @param <T> the type of the input to the function
  * @param <R> the type of the result of the function

src/main/java/spoon/reflect/visitor/chain/CtQuery.java

@@ -16,56 +16,124 @@
  */
 package spoon.reflect.visitor.chain;
 
+import spoon.reflect.declaration.CtElement;
 import spoon.reflect.visitor.Filter;
 
 import java.util.List;
 
 /**
  * CtQuery represents a query, which can be used to traverse a spoon model and collect
- * children elements in several ways.<br>
+ * children elements in several ways.<br/>
  *
- * <br>
- * Use {@link CtQueryable#map(CtFunction)}} or {@link CtQueryable#filterChildren(Filter)} to create a new query starting from an existing element.<br>
+ * Creation: A query is created either from a {@link CtElement}, or it can be defined first from {@link CtQueryImpl} and bound to root elements
+ * afterwards using {@link CtQuery#setInput(Object...)}.<br/>
  *
- * The main methods are:
- * <ul>
- * <li> {@link CtQueryable#map(CtFunction)} - uses a lambda expression to return any model elements that are directly accessible from an input element.
- * <li> {@link CtQueryable#filterChildren(Filter)} - uses {@link Filter} instances to filter children of an element
- * <li> {@link #list()} - to evaluate the query and return a list of elements produced by this query.
- * </ul>
- * The query can be used several times.<br>
- * A CtQuery  is lazily evaluated once {{@link #list()}} is called.
- * Usually a new query is created each time when one needs to query something.
- * However, reusing a {@link CtQuery} instance makes sense when the same query has to be evaluated
- * several times in a loop.
+ * Chaining: In a query several steps can be chained, by chaining calls to map functions. The non-null outputs of one step
+ * are given as input to the next step. An iterable or array output is considered as a set of different inputs for the next step.
+ *
+ * Evaluation: A CtQuery is lazily evaluated once {@link CtQuery#list()} or {@link CtQuery#forEach(CtConsumer)} are called.<br/>
  *
- * @param <O> the type of the element produced by this query
  */
-public interface CtQuery<O> extends CtQueryable {
+public interface CtQuery extends CtQueryable {
+
+	/**
+	 * Recursively scans all children elements of an input element.
+	 * The matched child element for which (filter.matches(element)==true) are sent to the next query step.
+	 * Essentially the same as {@link CtElement#getElements(Filter)} but more powerful, because it
+	 * can be chained with other subsequent queries.
+	 *
+	 * Note: the input element (the root of the query, `this` if you're in {@link CtElement}) is also checked and may thus be also sent to the next step.
+	 * The elements which throw {@link ClassCastException} during {@link Filter#matches(CtElement)}
+	 * are considered as **not matching**, ie. are excluded.
+	 *
+	 * @param filter used to filter scanned children elements of the AST tree
+	 * @return this to support fluent API
+	 */
+	@Override
+	<R extends CtElement> CtQuery filterChildren(Filter<R> filter);
+
+
+	/**
+	 * Query elements based on a function, the behavior depends on the return type of the function.
+	 * <table>
+	 * <tr><td><b>Return type of `function`</b><td><b>Behavior</b>
+	 * <tr><td>{@link Boolean}<td>Select elements if the returned value of `function` is true (as for {@link Filter}).
+	 * <tr><td>? extends {@link Object}<td>Send the returned value of `function` to the next step
+	 * <tr><td>{@link Iterable}<td>Send each item of the collection to the next step
+	 * <tr><td>{@link Object[]}<td>Send each item of the array to the next step
+	 * </table><br>
+	 *
+	 * @param function a Function with one parameter of type I returning a value of type R
+	 * @return this to support fluent API
+	 */
+	@Override
+	<I, R> CtQuery map(CtFunction<I, R> function);
+
+	/**
+	 * Sets (binds) the input of the query.
+	 * If the query is created by {@link CtElement#map} or {@link CtElement#filterChildren(Filter)},
+	 * then the query is already bound to this element.
+	 * A new call of {@link CtQuery#setInput(Object...)} is always possible, it resets the current binding and sets the new one.
+	 *
+	 * @param input
+	 * @return this to support fluent API
+	 */
+	<T extends CtQuery> T setInput(Object... input);
+
+	/**
+	 * Actually evaluates the query and for each produced output element of the last step,
+	 * calls `consumer.accept(outputElement)`.
+	 *
+	 * This avoids to create useless intermediate lists.
+	 *
+     * @param consumer The consumer which accepts the results of the query
+	 */
+	<R> void forEach(CtConsumer<R> consumer);
+
+	/**
+	 * Actually evaluates the query and returns all the elements produced in the last step.<br>
+	 * Note: The type R of the list is not checked by the query. So use the type, which matches the results of your query,
+	 * otherwise the ClassCastException will be thrown when reading the list.
+	 * @return the list of elements collected by the query.
+	 * @see #forEach(CtConsumer) for an efficient way of manipulating the elements without creating an intermediate list.
+	 */
+	<R extends Object> List<R> list();
 
 	/**
-	 * actually evaluates the query and returns all the produced elements collected in a List
+	 * Same as {@link CtQuery#list()}, but with static typing on the return type
+	 * and the final filtering, which matches only results, which are assignable from that return type.
+	 *
 	 * @return the list of elements collected by the query.
 	 */
-	List<O> list();
+	<R> List<R> list(Class<R> itemClass);
 
 	/**
 	 * Defines whether this query will throw {@link ClassCastException}
 	 * when the output of the previous step cannot be cast to type of input of next step.
-	 * The default value is {@link QueryFailurePolicy#FAIL}<br>
+	 * The default value is {@link QueryFailurePolicy#FAIL}, which means than exception is thrown when there is a mismatch<br>
 	 *
 	 * Note: The {@link CtQueryable#filterChildren(Filter)} step never throws {@link ClassCastException}
 	 *
 	 * @param policy the policy
 	 * @return this to support fluent API
 	 */
-	CtQuery<O> failurePolicy(QueryFailurePolicy policy);
+	CtQuery failurePolicy(QueryFailurePolicy policy);
 
 	/**
 	 * Sets the name of current query, to identify the current step during debugging of a query
 	 * @param name
 	 * @return this to support fluent API
 	 */
-	CtQuery<O> name(String name);
+	CtQuery name(String name);
 
+	/**
+	 * Same as {@link CtQuery#map(CtFunction)}, but the returned object is not handled
+	 * by java's return statement, but by a call to {@link CtConsumer#accept(Object)}, this
+	 * allows efficient and easy to write chained processing, see {@link CtConsumableFunction}.
+	 *
+	 * @param queryStep
+	 * @return this to support fluent API
+	 */
+	@Override
+	<I> CtQuery map(CtConsumableFunction<I> queryStep);
 }