Skip to content
Browse files

release commit for Pipes 1.0.

  • Loading branch information...
1 parent af10766 commit 62551c087180a30f4647fb0a852d46f002a21eaf @okram okram committed Feb 28, 2012
View
4 CHANGELOG.textile
@@ -5,13 +5,13 @@ Pipes: A Data Flow Framework using Process Graphs
!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-pipes.png!
-h3. Version 1.0 (Pipes -- NOT OFFICIALLY RELEASED YET)
+h3. Version 1.0 (Pipes -- February 28, 2012)
```xml
<dependency>
<groupId>com.tinkerpop</groupId>
<artifactId>pipes</artifactId>
- <version>1.0-SNAPSHOT</version>
+ <version>1.0</version>
</dependency>
```
View
10 doc/wiki/Acknowledgments.textile
@@ -0,0 +1,10 @@
+<img width="100" src="https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-character-2.png"/>
+
+This section provides a list of the people that have contributed in some way to the creation of Pipes.
+
+# "Marko A. Rodriguez":http://markorodriguez.com -- designed, developed, tested, and documented Pipes.
+# "Darrick Weibe":http://github.com/pangloss -- implemented Pipe paths.
+# "Zach Cox":http://theza.ch - helped develop the fluent model.
+# "Ketrina Yim":http://www.ketrinayim.com/ -- designed the Pipes logo.
+
+Please review Pipes' "pom.xml":http://github.com/tinkerpop/pipes/blob/master/pom.xml. Pipes would not be possible without the work done by others to create these useful packages.
View
52 doc/wiki/Basic-Pipes.textile
@@ -0,0 +1,52 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-plumber.png!
+
+A @Pipe<S,E>@ is a Java interface that extends both the @Iterable<E>@ and @Iterator<E>@ interface. A @Pipe<S,E>@ takes, as input, an iterator or iterable yielding objects of type @S@ and produces/emits objects of type @E@. The character "S" stands for "starts" and the character "E" stands for "ends".
+
+Here is a simple example demonstrating a single pipe that capitalizes the characters of the strings that come into it.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+capsPipe.setStarts(Arrays.asList("this", "is", "the", "end."));
+while(capsPipe.hasNext()) {
+ System.out.print(capPipe.next() + " ");
+}
+```
+
+This pipe will produce the following output.
+
+bc. THIS IS THE END.
+
+Given that @Pipe<S,E>@ extends @Iterator<E>@ and @Iterable<E>@, its possible to string together pipes to create a processing line.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+Pipe<String,Integer> countPipe = new CountPipe();
+capsPipe.setStarts(Arrays.asList("this", "is", "the", "end."));
+countPipe.setStarts(capsPipe);
+while(countPipe.hasNext()) {
+ System.out.print(countPipe.next() + " ");
+}
+```
+
+If @CountPipe@ takes a @String@ and emits the number of characters in that @String@, then the previous code will yield the following output.
+
+bc. 4 2 3 4
+
+Realize that the output of one pipe must be of the same type as the input of the next pipe. Given that @Pipe<S,E>@ extends @Iterator<E>@, The @E@ of the first pipe becomes the @S@ of the second pipe. In order to make it easier to create chains of pipes, there is a handy @Pipeline<S,E>@ class. This class implements @Pipe<S,E>@ and thus, @Pipeline<S,E>@ objects can be combined like any other pipe (i.e. you can create pipelines of pipelines). Here is an example using a @Pipeline<S,E>@ object.
+
+```java
+Pipe<String,String> capsPipe = new CapitalizePipe();
+Pipe<String,Integer> countPipe = new CountPipe();
+Pipe<Integer,String> wordPipe = new WordPipe();
+Pipeline<String,String> pipeline = new Pipeline<String,String>(capsPipe, countPipe, wordPipe);
+pipeline.setStarts(Arrays.asList("this", "is", "the", "end."));
+while(pipeline.hasNext()) {
+ System.out.print(pipeline.next() + " ");
+}
+```
+
+Assuming that @WordPipe@ emits the word version of an incoming integer, the pipeline will produce the following output.
+
+bc. four two three four
+
+That's Pipes in a nutshell.
View
54 doc/wiki/Creating-a-Pipe.textile
@@ -0,0 +1,54 @@
+[[https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-3.png]]
+
+The pipes that come prepackaged with Pipes are not sufficient for every use case. In many situations, it will be necessary to create a pipe that will solve a particular mapping from an input to an output. An easy way to accomplish this is via the respective @XXXClosurePipe@ and @PipeClosure@ implementation. However, it is possible to write a specific pipe without using @XXXClosurePipe@-based pipes.
+
+h2. Transform-Based Pipe with AbstractPipe
+
+Pipes make it easy to create new pipes with the @AbstractPipe@ class. In @AbstractPipe@, there is a single method that needs to be implemented: @AbstractPipe.processNextStart()@. The example below demonstrates the internals of a simple pipe that maps a string to the number of characters contained in that string.
+
+```java
+public class WordLengthPipe extends AbstractPipe<String, Integer> {
+ public Integer processNextStart() {
+ String start = this.starts.next();
+ return start.length();
+ }
+}
+```
+
+h2. Filter-Based Pipe with AbstractPipe
+
+The general pattern for filter-based pipes is explained with an example.
+
+```java
+public class WordFilterPipe extends AbstractPipe<String,String> implements FilterPipe<String> {
+ public String processNextStarts() {
+ while(true) {
+ String start = this.starts.next();
+ if(start.length() > 4)
+ return start;
+ }
+ }
+}
+```
+
+A @FilterPipe@ will usually make use of a @while(true)@ loop that will continually pull in new objects. If the current object meets some criteria, it will be emitted, else, the @while(true)@ will loop again and pull in a new object. There is no need to worry about handling @NoSuchElementExceptions@. If @this.starts.next()@ throws a @NoSuchElementException@ then this will be handled appropriately by @AbstractPipe@.
+
+h2. SideEffect-Based Pipe with AbstractPipe
+
+Below is an example of a side-effect based pipe using @AbstractPipe@.
+
+```java
+public class WordPrintPipe extends AbstractPipe<String,String> implements SideEffectPipe<String,Object> {
+ public String processNextStarts() {
+ String start = this.starts.next();
+ System.out.println(start);
+ return start;
+ }
+
+ public Object getSideEffect() {
+ return null;
+ }
+}
+```
+
+Given that the side-effect is simply a @println@ to the @System.out@, there is no @Object@ side-effect that can be later retrieved. As such, the @SideEffectPipe.getSideEffect()@ method simply returns @null@.
View
109 doc/wiki/Filter-Pipes.textile
@@ -0,0 +1,109 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/duck-plumber.jpg!
+
+A common Pipes pattern is the filtering of objects. A filter-based pipe will consume objects and either emit them or not. If they are not emitted, then they are considered filtered by the pipe. A useful interface to implement that describes this behavior is @FilterPipe<S> extends Pipe<S,S>@.
+
+h2. Generic Filter Pipe
+
+The generic filter pipe is @FilterFunctionPipe@. A @FilterFunctionPipe@ takes a @PipeFunction@ (see [[Pipe Types]]) that computes on @S@ and either emits it or not. An example @PipeFunction@ is provided below:
+
+```java
+public class CharCountPipeFunction implements PipeFunction<String,Boolean> {
+
+ private final int number;
+
+ public CharCountPipeFunction(int number) {
+ this.number = number;
+ }
+
+ public Boolean compute(String argument) {
+ return argument.length() == this.number;
+ }
+}
+```
+
+When put in the context of a @FilterFunctionPipe@, the code looks as follows:
+
+```java
+Pipe<String, String> pipe = new FilterFunctionPipe<String>(new CharCountPipeFunction(4));
+pipe.setStarts(Arrays.asList("tell", "me", "your", "name"));
+// the results of the iteration are: "tell", "your", "name"
+```
+
+h2. Basic Filtering
+
+The @RandomFilterPipe@ comes with the Pipes distribution. @RandomFilterPipe@ will only allow a consumed object to be emitted if a biased coin toss lands on "heads." At the extremes, if @bias@ is 0.0 then no incoming objects are emitted and if @bias@ is 1.0, then every incoming object is emitted.
+
+```java
+public class RandomFilterPipe<S> extends AbstractPipe<S, S> implements FilterPipe<S> {
+ private static final Random RANDOM = new Random();
+ private final double bias;
+ public SampleFilterPipe(final double bias) {
+ this.bias = bias;
+ }
+ protected S processNextStart() {
+ while (this.starts.hasNext()) {
+ S s = this.starts.next();
+ if (bias >= RANDOM.nextDouble()) {
+ return s;
+ }
+ }
+ throw new NoSuchElementException();
+ }
+}
+```
+
+The @processNextStart()@ method structure above is a common pattern in filter-based pipes. In short, a @while(this.starts.hasNext())@ is evaluated until an incoming start object meets the criteria and is returned (i.e. emitted). If there are no more starts and the criteria is not met, then a @NoSuchElementException@ is thrown. Beware of using recursion to accomplish a similar behavior. As the amount of data grows that is streaming through, recursion-based methods can easily yield a @StackOverflowError@.
+
+h2. Comparison-Based Filtering
+
+More complicated filters can be created than what was presented above. These filters usually check objects to determine whether to emit them or not. There is an interface called @ComparisonFilterPipe<S,T>@. Implementations of this interface consume objects of type @S@. If a check of that object passes (i.e. return @true@), then the @S@ object is emitted, else it is filtered. @ComparisonFilterPipe@ has the following signature:
+
+```java
+public interface ComparisonFilterPipe<S, T> extends FilterPipe<S> {
+ public enum Filter {
+ EQUAL, NOT_EQUAL, GREATER_THAN, LESS_THAN, GREATER_THAN_EQUAL, LESS_THAN_EQUAL
+ }
+ public boolean compareObjects(T leftObject, T rightObject);
+}
+```
+
+The important method is @compareObjects()@. This method returns @true@ if the left hand object is @EQUAL@, @NOT_EQUAL@, etc. to the right hand object.
+
+Next, there is an abstract class called @AbstractComparisonFilterPipe<S,T>@ that implements @ComparisonFilterPipe@. More specifically, it provides a standard implementation of @ComparisonFilterPipe.compareObjects()@. For most situations, this method is sufficient. However, if not, simply override the method with an implementation that meets the requirements of the designed pipe.
+
+```java
+public boolean compareObjects(final T leftObject, final T rightObject) {
+ switch (this.filter) {
+ case EQUAL:
+ if (null == leftObject)
+ return rightObject == null;
+ return leftObject.equals(rightObject);
+ case NOT_EQUAL:
+ if (null == leftObject)
+ return rightObject != null;
+ return !leftObject.equals(rightObject);
+ case GREATER_THAN:
+ if (null == leftObject || rightObject == null)
+ return false;
+ return ((Comparable) leftObject).compareTo(rightObject) == 1;
+ case LESS_THAN:
+ if (null == leftObject || rightObject == null)
+ return false;
+ return ((Comparable) leftObject).compareTo(rightObject) == -1;
+ case GREATER_THAN_EQUAL:
+ if (null == leftObject || rightObject == null)
+ return false;
+ return ((Comparable) leftObject).compareTo(rightObject) >= 0;
+ case LESS_THAN_EQUAL:
+ if (null == leftObject || rightObject == null)
+ return false;
+ return ((Comparable) leftObject).compareTo(rightObject) <= 0;
+ default:
+ throw new RuntimeException("Invalid state as no valid filter was provided");
+ }
+ }
+```
+
+Note: it many situations, such as @ObjectFilterPipe@, the right hand object to allow or disallow is stored in the pipe and compared with each object passed through it.
+
+!http://github.com/tinkerpop/pipes/raw/master/doc/images/filter-example.png!
View
63 doc/wiki/Home.textile
@@ -0,0 +1,63 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-logo.png!
+
+Pipes is a "dataflow":http://en.wikipedia.org/wiki/Dataflow_programming framework using "process graphs":http://en.wikipedia.org/wiki/Kahn_process_networks. A process graph is composed of @Pipe@ vertices connected by communication edges. A @Pipe@ implements a simple computational step that can be composed with other @Pipe@ objects to create a larger computation. Such data flow graphs allow for the splitting, merging, looping, and in general, the transformation of data from input to output.
+
+There are numerous Pipe classes that come with the main Pipes distribution. Once a good understanding of each Pipe accomplished, then using the framework is straightforward. In general, the best way to learn about all the Pipes provided is through the project JavaDoc.
+
+Please join the Gremlin users group at "http://groups.google.com/group/gremlin-users":http://groups.google.com/group/gremlin-users for all "TinkerPop":http://tinkerpop.com related discussions.
+
+Pipes 0.9 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.9/api/
+Pipes 0.8 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.8/api/
+Pipes 0.7 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.7/api/
+Pipes 0.6 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.6/api/
+Pipes 0.5 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.5/api/
+Pipes 0.4 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.4/api/
+Pipes 0.3 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.3/api/
+Pipes 0.2 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.2/api/
+Pipes 0.1 "JavaDoc":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.1/api/
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>1.0</version>
+</dependency>
+```
+
+```java
+Pipe<String,String> pipe1 = new RemoveCharacterPipe("o");
+Pipe<String,Integer> pipe2 = new CountCharactersPipe();
+Pipe<String,Integer> pipeline = new Pipeline<String,Integer>(pipe1, pipe2);
+pipeline.setStarts(Arrays.asList("marko","josh","peter"));
+for(Integer number : pipeline) {
+ System.out.println(number);
+}
+
+4
+3
+5
+```
+
+"On the Nature of Pipes":http://markorodriguez.com/2011/08/03/on-the-nature-of-pipes/ (*Graphical Presentation of the Pipe Mechanics*)
+
+==<hr/>==
+
+* Basic Concepts
+ ** [[Basic Pipes]]
+ ** [[Pipe Types]]
+ *** [[Transform Pipes]]
+ *** [[Filter Pipes]]
+ *** [[SideEffect Pipes]]
+ ** Pipe SubTypes
+ *** [[MetaPipes]]
+ *** Branch Pipes
+* Advanced Concepts
+ ** [[Creating a Pipe]]
+ ** [[Transformation Paths]]
+* Conclusion
+ ** [[Acknowledgments]]
+ ** [[Release Notes]]
+
+==<hr/>==
+
+fn1. Pipes documentation is up to date with the current Pipes "codebase":http://github.com/tinkerpop/pipes/tree/master, not with the latest Pipes "release":http://github.com/tinkerpop/pipes/downloads.
View
52 doc/wiki/MetaPipes.textile
@@ -0,0 +1,52 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-3.png!
+
+A metapipe is a pipe that "wraps" another pipe. MetaPipes are useful in that they can make decisions based upon the behavior of the pipe(s) they wrap. There are numerous metapipes and this section will discuss a few of the more interesting cases.
+
+h2. Basic Pattern
+
+In many situations, the basic pattern used when creating a metapipe is to have the constructor of the metapipe take @n@ number of pipes. Then for each incoming @S@ object to the metapipe, it pushes that @S@ object to its @n@ pipes and makes a emit decision based upon how the pipes behave. In many cases, you can think of a metapipe as an "overseer" of its internal pipes. This idea is presented in the typical code pattern found in metapipes.
+
+```java
+public E processNextStart() {
+ S s = this.starts.next();
+ for (Pipe<S, E> pipe : this.pipes) {
+ pipe.setStarts(new SingleIterator<S>(s));
+ E e = pipe.next();
+ // decide about what to do given the output of the internal pipe
+ }
+ // if all pipes behave as desired, perhaps do:
+ return s.getE();
+}
+```
+
+@SingleIterator<S>@ is a handy class that wraps an object for a single legal @next()@. This class is much more efficient than doing @Arrays.asList(s).iterator()@.
+
+h2. Pipelines
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipeline-example.png!
+
+A @Pipeline@ is a commonly used pipe. A @Pipeline<S,E>@ implements @Pipe<S,E>@ and as such, a @Pipeline@ is simply a @Pipe@. A @Pipeline@ takes an ordered list of pipes in its constructor. It connects these pipes, whereby the input of pipe @n@ is connected to the output of pipe @n-1@. Note that the output type of pipe @n-1@ must be the same type as the input to pipe @n@. This idea is elucidated in the diagram above.
+
+The benefit of using a @Pipeline@ is that is greatly reduces the complexity of a process and allows for easy reuse. It is analogous, in many ways, to creating a function to wrap a body of code/instructions. As such, the function serves as a blackbox with input and output.
+
+h2. And/Or Pipes
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/andfilterpipe-example.png!
+
+There are two logical @FilterPipe@ pipes called @AndFilterPipe<S>@ and @OrFilterPipe<S>@. These pipes take an object of type @S@ and emit an object of type @S@. However, they only emit the @S@ object if the collection of @Pipe<S,Boolean>@ pipes that they wrap return @true@. For @AndFilterPipe@ to emit its incoming @S@ object, all of its wrapped pipes must return @true@ that @S@ object. For the @OrFilterPipe@, only one of its wrapped pipes must return @true@. If you want to see if a number is greater than or equal to 1 *and* less than 10, then use the following @AndFilterPipe@.
+
+```java
+Pipe<Integer,Integer> pipeA = new ObjectFilterPipe<Integer>(1, ComparisonFilterPipe.GREATER_THAN_EQUAL);
+Pipe<Integer,Integer> pipeB = new ObjectFilterPipe<Integer>(10, ComparisonFilterPipe.LESS_THAN);
+Pipe<Integer,Integer> pipe1 = new AndFilterPipe<Integer>(new HasNextPipe<Integer>(pipeA), new HasNextPipe<Integer>(pipeB));
+pipe1.setStarts(Arrays.asList(1,22,10,136,7,2,67));
+while(pipe1.hasNext()) {
+ System.out.print(pipe1.next() + "...");
+}
+```
+
+The @System.out@ of the previous code is as follows.
+
+bc. 1...7...2...
+
+Note the use of @HasNextPipe<S>@ which implements @Pipe<S,Boolean>@. If the @ObjectFilterPipe@ pipes that compose the @AndFilterPipe@ filter their incoming @S@ object, then they will return false on a call to @hasNext()@. Thus, @HasNextPipe@ is useful for determining if a filter pipe has filtered an object.
View
22 doc/wiki/Pipe-Types.textile
@@ -0,0 +1,22 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-3.png!
+
+There are 4 types of pipes.
+ # *Transform*: take an object of type @S@ and emit an object of type @E@.
+ ** @Pipe<S,E>@
+ # *Filter*: take an object of type @S@ and either emit it or not.
+ ** @FilterPipe<S> extends Pipe<S,S>@
+ # *Side-Effect*: take an object of type @S@, perform some computation, emit the same object.
+ ** @SideEffectPipe<S,T> extends Pipe<S,S>@
+ # *Branch*: take an object and select any number of paths to send it down.
+
+The packaging of Pipes is set up such that there is a @transform@, @filter@, @sideeffect@, and @branch@ package. All pipes are instances of one of these 4 types. Moreover, each type has a special "function"-version known as a @XXXFunctionPipe@. For example: @TransformFunctionPipe@, @FilterFunctionPipe@, and @SideEffectFunctionPipe@. These pipes are discussed in their respective sections of the documentation. However, the general description of a @PipeFunction@ is presented next.
+
+h2. Pipe Function
+
+Pipes has an interface called @PipeFunction<A,B>@ that has the following methods:
+
+```java
+public B compute(A argument);
+```
+
+A pipe that takes a @PipeFunction@ will use the provided @compute()@ method at a particular point according to the semantics of the pipe. Thus, the JavaDoc of such pipes will articulate the meaning/use of the provided @PipeFunction@.
View
263 doc/wiki/Release-Notes.textile
@@ -0,0 +1,263 @@
+h2. Pipes 1.X
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-pipes.png!
+
+h3. Version 1.0 (Pipes -- February 28, 2012)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>1.0</version>
+</dependency>
+```
+
+* Added @SelectPipe@ to select the objects of named steps from a pipeline
+* Separated @Row@ from @Table@ so it can be reused by other pipes (e.g. @SelectPipe@)
+* Heavily reworked @PipesFluentPipeline@ JavaDoc to make it a single point of reference
+* Updated @LoopPipe@ semantics such that if there is an emit function, then the final object in a while isn't returned
+* Updated @GroupCountFunctionPipe@ semantics such that the value-function gets a @Pair@ containing incoming object and last value for that object
+* Reorganized the @com.tinkerpop.pipes.util@ package
+* Removed the @objectFilter@ step from @PipeFluentPipeline@ as @retain@ and @except@ can be used instead
+* @EmptyIterator@ is now a singleton
+* @ScatterPipe@ now scatters @Map@ objects into @Map.Entry@ streams
+* Added @GroupByPipe@ which groups values under a provided key
+* Added @GroupByReducePipe@ which groups reduced values under a provided key
+
+h2. Pipes 0.X
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-sink-pipe.png!
+
+h3. Version 0.9 (Sink -- December 7, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.9</version>
+</dependency>
+```
+
+Pipes 0.9 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.9/api/
+
+* Removed all "Blueprints":http://blueprints.tinkerpop.com graph related dependencies
+* Renamed @UniquePathFilterPipe@ to @CyclicPathFilterPipe@
+* Renamed and removed numerous methods in @FluentPipeline@ to make DSL construction more concise
+* Updated @LoopPipe@ to support emit functionality via @PipeFunction@
+* Added @StorePipe@ to lazily save objects to provided collection
+* Added function support to @DeduplicateFilterPipe@
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-cleaner-pipe.png!
+
+h3. Version 0.8 (Cleaner -- September 18, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.8</version>
+</dependency>
+```
+
+Pipes 0.8 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.8/api/
+
+* @SideEffectPipe@ now has signature of @<S,T>@ instead of @<S,E,T>@
+* @GroupCountFunctionPipe@ has signature of @<S,K>@ to denote the key of its @Map<K,Number>@ side-effect
+* Removed @ComparisonFilterPipe@ as it is confusing and now @PipeHelper@ has comparison support
+* Added @PipeHelper.makePipeString()@ to have a single point where all pipe @toString()@s are created
+* Optimized @PipeHelper.counter()@ and @PipeHelper.fillCollection()@ by respecting architecture of @AbstractPipe@
+* Added @PipeHelper.iterate()@ to complete drain an iterator
+* Added @FluentPipeline@ as a utility class for fluently constructing a pipeline
+* Added @RetainFilterPipe@, added @ExceptFilterPipe@, and abstracted @CollectionFilterPipe@
+* Added @EdgesPipe@, added @VerticesPipe@, and abstracted @GraphElementPipe@
+* Renamed @AggregatorPipe@ to @AggregatePipe@
+* @AndFilterPipe@ and @OrFilterPipe@ will automatically wrap provided pipes in a @HasNextPipe@
+* Generalized @GroupCountPipe@ to have a value of @Map<Object,Number>@ instead of @Long@
+* Migrated @TablePipe@ and @Table@ from Gremlin
+* Simplified the semantics of @HasNextPipe@ (use @Pipe.hasNext()@ to accomplish previous behavior)
+* Added @MemoizePipe@ to memoize the mapping of a particular pipe
+* Simplified @PipeClosure@ to be purely functional and now named @PipeFunction@
+** Renamed @XXXClosurePipe@ to @XXXFunctionPipe@
+* Fixed performance issue in @RangeFilterPipe@ and provided @long@ ranges
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-pvc-pipe.png!
+
+h3. Version 0.7 (PVC -- August 1, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.7</version>
+</dependency>
+```
+
+Pipes 0.7 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.7/api/
+
+* Added support for @Vertex.getOutEdges(String...)@ and @Vertex.getInEdges(String...)@
+* Updated @AggregatorPipe@ to be fully @SideEffectPipe@ compliant and to respect @getPath()@
+* Added @PipeClosure@ to support closure-based pipes
+* Migrated all the "Gremlin":http://gremlin.tinkerpop.com specific pipes to Pipes
+* Refactored package names to align with pipe semantics
+* Flipped the meaning of @Filter@, where @true@ means pass, and @false@ means filter
+* Added @AggregatorPipe@ @PipeClosure@ to process object prior to insertion into the aggregate collection
+* Added temporary fix for @AggregatorPipe@/@LoopPipe@ phenomena
+* @IfThenElsePipe@ now requires an @elseClosure@
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-toilet-pipe.png!
+
+h3. Version 0.6 (Toilet -- June 15, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.6</version>
+</dependency>
+```
+
+Pipes 0.6 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.6/api/
+
+* @RangeFilterPipe@ max value is now inclusive
+* Added @OptionalPipe@ which is like @BackFilterPipe@, but non-filtering.
+* Updated @AggregatorPipe@ to support multiple @setStarts()@
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-drain-pipe.png!
+
+h3. Version 0.5 (Drain -- May 8, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.5</version>
+</dependency>
+```
+
+Pipes 0.5 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.5/api/
+
+* Added @OutPipe@, @InPipe@, and @BothPipe@ to allow for vertex-vertex hoping
+* Removed @VertexEdgeLabelFilterPipe@ as it is now covered by more efficient pipes
+* Added the @MetaPipe@ interface to allow pipes to acknowledge pipe wrapping
+* Added @CopySplitPipe@, @ExhaustMergePipe@, and @FairMergePipe@
+* Optimized @PropertyPipe@ and @UniquePathFilterPipe@
+* Added support for @Pipe.reset()@ and thus, pipe reuse
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-spigot-pipe.png!
+
+h3. Version 0.4 (Spigot -- April 4, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.4</version>
+</dependency>
+```
+
+Pipes 0.4 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.4/api/
+
+* Added @StartPipe@ to aid in creating an @Object@-to-@Pipe@ converter
+* Removed @VertexEdge@ and @EdgeVertex@ pipes for more low-level/faster pipes.
+* Supports new Blueprints packaging scheme.
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-mario-pipe.png!
+
+h3. Version 0.3.1 (Mario -- March 2, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.3.1</version>
+</dependency>
+```
+
+Pipes 0.3.1 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.3.1/api/
+
+* Added @VertexEdgeLabelFilterPipe@ to reflect Blueprints @Vertex@ API update
+* Added @UniquePathFilterPipe@ to filter paths that have visited the same object twice
+
+h3. Version 0.3 (Mario -- January 22, 2011)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.3</version>
+</dependency>
+```
+
+Pipes 0.3 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.3/api/
+
+* Added a general @toString()@ model to articulate pipe construction
+* Redesigned path-model to where @enablePaths()@ no longer required
+** Added @HistoryIterator@ to replace the @enablePath()@ model
+* Renamed @CountCombine@ to @GroupCount@
+* Added @ExpandableIterator@ to support pipe looping
+* Removed many unused pipes for the sake of clarity and simplicity
+* Added @GatherPipe@ as a composite of @SideEffectCapPipe@ and @AggregatorPipe@
+* Removed @EndSupportPipe@ as that is what @ScatterPipe@ now accomplishes
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-clogged-pipe.png!
+
+h3. Version 0.2 (Clogged -- December 14, 2010)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.2</version>
+</dependency>
+```
+
+Pipes 0.2 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.2/api/
+
+* Added path functionality to get the path pipe traversed
+** Added @PathSequence@ path utility
+** Added @PathPipe@ to support using paths in a computation
+* Numerous test cases added
+
+==<hr/>==
+
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/pipes-leaky-pipe.png!
+
+h3. Version 0.1.1 (Leaky -- October 28, 2010)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.1.1</version>
+</dependency>
+```
+
+* Fixed 'reset pipe' issue in @FutureFilterPipe@
+
+h3. Version 0.1 (Leaky -- September 22, 2010)
+
+```xml
+<dependency>
+ <groupId>com.tinkerpop</groupId>
+ <artifactId>pipes</artifactId>
+ <version>0.1</version>
+</dependency>
+```
+
+Pipes 0.1 "JavaDocs":http://tinkerpop.com/maven2/com/tinkerpop/pipes/0.1/api/
+
+* Initial release as the core engine for "Gremlin":http://gremlin.tinkerpop.com 0.5
View
33 doc/wiki/SideEffect-Pipes.textile
@@ -0,0 +1,33 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/duck-plumber.jpg!
+
+A side-effect based pipe will emit the incoming object unaltered. However, during that process, a computation is effected. This computation can be used to yield a computational side-effect.
+
+h2. Generic SideEffect Pipe
+
+The generic side-effect pipe is @SideEffectFunctionPipe@. A @SideEffectFunctionPipe@ takes a @PipeFunction@ (see [[Pipe Types]]) that computes on @S@ and emits @S@. However, the provided @PipeFunction@ provides a side-effect computation. An example @PipeFunction@ is provided below:
+
+```java
+public class CharTotalPipeFunction implements PipeFunction<String, Object> {
+
+ private final AtomicInteger count;
+
+ public CharCountPipeFunction(AtomicInteger count) {
+ this.count = count;
+ }
+
+ public Object compute(String argument) {
+ this.count.getAndAdd(argument.length())
+ return null;
+ }
+}
+```
+
+When put in the context of a @SideEffectClosurePipe@, the code looks as follows:
+
+```java
+AtomicInteger count = new AtomicInteger(0);
+Pipe<String, String> pipe = new SideEffectFunctionPipe<String>(new CharTotalPipeFunction(count));
+pipe.setStarts(Arrays.asList("tell", "me", "your", "name"));
+// the results of the iteration are: "tell", "me", "your", "name"
+assert count.get() == 14;
+```
View
23 doc/wiki/Transform-Pipes.textile
@@ -0,0 +1,23 @@
+!https://github.com/tinkerpop/pipes/raw/master/doc/images/duck-plumber.jpg!
+
+Transform pipes are used to take an object of type @S@ and emit an object of type @E@. While the @S@ is generally not altered, the term "transform" refers to the fact that from @S@, @E@ is yielded.
+
+h2. Generic Transform Pipe
+
+The generic transform pipe is @TransformFunctionPipe@. A @TransformFunctionPipe@ takes a @PipeFunction@ (see [[Pipe Types]]) that computes on @S@ and emits the @E@ that the computation returned. An example @PipeFunction@ is provided below:
+
+```java
+public class NumCharsPipeFunction implements PipeFunction<String,Integer> {
+ public Integer compute(String argument) {
+ return argument.length();
+ }
+}
+```
+
+When put in the context of a @TransformFunctionPipe@, the code looks as follows:
+
+```java
+Pipe<String,Integer> pipe = new TransformFunctionPipe<String,Integer>(new NumCharsPipeFunction());
+pipe.setStarts(Arrays.asList("tell", "me", "your", "name"));
+// the results of the iteration are: 4, 2, 4, 4
+```
View
35 doc/wiki/Transformation-Paths.textile
@@ -0,0 +1,35 @@
+[[https://github.com/tinkerpop/pipes/raw/master/doc/images/plumber-2.png]]
+
+Pipes supports the notion of transformational paths (see @Pipe.getPath()@). It is possible to know how an object at the end a Pipe was transformed through all the stages of the Pipe.
+
+```java
+Pipe<Object, Object> pipeline = new Pipeline<Object,Object>(...);
+pipeline.setStarts(new SingleIterator<Object>(o));
+while(pipeline.hasNext()) {
+ Object object = pipeline.next();
+ List<Object> path = pipeline.getPath();
+}
+```
+
+h2. Path Example
+
+For example, assume a @Pipeline@ with the following self-describing pipes: @NumCharsPipe@, @NumToWordPipe@, @NumCharsPipe@.
+
+```java
+Pipeline<String,Integer> pipeline = new Pipeline(new NumCharsPipe(), new NumToWordPipe(), new NumCharsPipe());
+pipeline.setStarts(Arrays.asList("lady","in","red"));
+while(pipeline.hasNext()) {
+ pipeline.next();
+ System.out.println(pipeline.getPath());
+}
+```
+
+The @System.out@ yields:
+
+```text
+[lady, 4, four, 4]
+[in, 2, two, 3]
+[red, 3, three, 5]
+```
+
+*NOTE*: See @PathPipe<S> implements Pipe<S,List>@ for a pipe that takes in an object and emits the path thus far.
View
2 pom.xml
@@ -10,7 +10,7 @@
</parent>
<groupId>com.tinkerpop</groupId>
<artifactId>pipes</artifactId>
- <version>1.0-SNAPSHOT</version>
+ <version>1.0</version>
<packaging>jar</packaging>
<url>http://pipes.tinkerpop.com</url>
<name>Pipes: A Lazy Data Flow Framework</name>

0 comments on commit 62551c0

Please sign in to comment.
Something went wrong with that request. Please try again.