Modularization

#Modularization how-to

##Basics

The basic interface that all of the processing components should have in common should be modularization.Module. However, since most of the methods required by that interface are implemented within modularization.ModuleImpl, it is more convenient to just let your module extend that abstract class.

If you choose this way, you will have minimal work and only have to override the following methods:
public boolean process() --- the part where the actual processing takes place.
protected void applyProperties() --- this is in fact only needed if your module has own properties that can be applied.

Input and output of the modules are done through pipes that are capsuled into classes, namely modularization.BytePipe and CharPipe. The former hosts both a PipedInputStream and PipedOutputStream (connected to each other), the latter PipedReader and PipedWriter respectively.

Each module has one input pipe and as many output pipes as needed. However you will have to define which I/O your module supports within the constructor by adding the respective classes to the internal lists, e.g.

this.getSupportedInputs().add(CharPipe.class);
this.getSupportedOutputs().add(CharPipe.class);

Also in your constructor, you can add meta information about the module and its properties, e.g.

// Add default values
this.getPropertyDefaultValues().put(ModuleImpl.PROPERTYKEY_NAME, "AtomicRangeSuffixTreeBuilder");
this.getPropertyDefaultValues().put(PROPERTYKEY_MAXLENGTH, "10");

// Add module description
this.setDescription("Iterates over a raw and unsegmented string input, building a suffix tree from the data of limited range with each step. Keeps track of how often each node of the suffix tree gets triggered.");

// Add description for properties
this.getPropertyDescriptions().put(PROPERTYKEY_MAXLENGTH,"Define the maximum length of any branch of the tree.");

Adding this information is in fact mandatory because the discovery of a module's accepted properties is done through its list of property descriptions.

I/O

In the process method, you will need to read input data and write your results to output; here's how you can do it smoothly:

Read

You can either use the read(...) methods this.getInputCharPipe() and this.getInputBytePipe() provide, or you can access the Piped…--classes directly, like this for example (this parses a file array from the input using the Google gson library):

inputFileList = gson.fromJson(this.getInputCharPipe().getInput(), new File[0].getClass());

Write

Since there can be many output pipes attached, you will best use the this.outputToAllCharPipes(String data) or this.outputToAllBytePipes(byte[] data) method. It will write your data to all outputs in a performance-conscious manner.

Examples

For further reference, do have a look at modularization.ExampleModule and the basic modules.

Using modules

Modules are meant to be connected via Pipes and started in parallel, so that the processed data trickles from module to module. Details follow below.

Parallelization

The Module interface extends the interface parallelization.CallbackProcess (see Parallelization), but all necessary methods are already implemented within modularization.ModuleImpl so you can use modules that extend that class in conjunction with a callback receiver without any additional work.

Module trees

To facilitate the whole module-pipe-anothermodule business, the class modularization.ModuleTree exists. It keeps track of an arbitrary number of modules that can be arranged in a tree-like structure, meaning one module has one input but as many outputs as needed that can connect to other modules inputs.

Probably the best way to get an overview of how to handle module trees is to have a look at the class modularization.ModuleTreeTest -- it shows the basic usage of the modularization.ModuleTree class.