PiCo stands for Pipeline Composition.
The main entity in PiCo is the Pipeline, basically a graph-composition of processing elements.
This model is intended to give the user an unique interface for both stream and batch processing, hiding completely data management and focusing only on operations, which are represented by Pipeline stages.
The DSL we propose is entirely implemented in C++11, exploiting the FastFlow library as runtime.
How to compile
To build any PiCo application, the FastFlow library should be provided. FastFlow can be downloaded from the GitHub repository and then add a symbolic link to the FastFlow library into the PiCo directory:
$ ln -s /path/to/FastFlow/ff /PiCo/root/dir
examples directory can be found an optimized word count, a stock-market and a page rank benchmarks.
The word count example can be executed as the following:
$ cd word-count
$ ./pico_wc <input file> <output file>
and the input data can be generated with the application in the
Input text is generated by utilizing an english dictionary
An input text can be generated as follows:
$ cd testdata
$ ./generate_text <dictionary file> <n. of lines> >> <output.file>
the parallel degree can be set manually with the environment variable PARDEG.
Each example produces a
.dot file for a graphical representation of the application pipeline coded in Graphviz.
To visualize these graphs, use the command:
dot -Tpng filename.dot -o outfile.png