Modularization

#Modularization how-to

##Basics

The basic interface that all of the processing components should have in common should be modularization.Module. However, since most of the methods required by that interface are implemented within modularization.ModuleImpl, it is more convenient to just let your module extend that abstract class.

If you choose this way, you will have minimal work and only have to override the following methods: public boolean process() --- the part where the actual processing takes place. protected void applyProperties() --- this is in fact only needed if your module has own properties that can be applied.

Input and output of the modules are done through pipes that are capsuled into classes, namely modularization.BytePipe and CharPipe. The former hosts both a PipedInputStream and PipedOutputStream (connected to each other), the latter PipedReader and PipedWriter respectively.

Each module has one input pipe and as many output pipes as needed. However you will have to define which I/O your module supports within the constructor by adding the respective classes to the internal lists, e.g.

this.getSupportedInputs().add(CharPipe.class);
this.getSupportedOutputs().add(CharPipe.class);

Also in your constructor, you can add a handy description of the properties your module accepts (but this is optional), e.g.

this.getPropertyDescriptions().put(PROPERTYKEY_ADDSTARTSYMBOL, "Set to 'true' if '"+STARTSYMBOL+"' should be added as start symbol to each sentence");
this.getPropertyDescriptions().put(PROPERTYKEY_ADDTERMINALSYMBOL, "Set to 'true' if '"+TERMINIERSYMBOL+"' should be added as start symbol to each sentence");
this.getPropertyDescriptions().put(PROPERTYKEY_CONVERTTOLOWERCASE,"If set to 'true' the output will be all lowercase");
this.getPropertyDescriptions().put(PROPERTYKEY_KEEPPUNCTUATION,"If set to 'true' punctuation will not be discarded");
this.getPropertyDescriptions().put(PROPERTYKEY_OUTPUTANNOTATEDJSON,"If set to 'true' the output will be annotated JSON instead of plain text");

I/O

In the process method, you will need to read input data and write your results to output; here's how you can do it smoothly:

Read

You can either use the read(...) methods this.getInputCharPipe() and this.getInputBytePipe() provide, or you can access the Piped…--classes directly, like this for example (this parses a file array from the input using the Google gson library):

inputFileList = gson.fromJson(this.getInputCharPipe().getInput(), new File[0].getClass());

Write

Since there can be many output pipes attached, you will best use the this.outputToAllCharPipes(String data) or this.outputToAllBytePipes(byte[] data) method. It will write your data to all outputs in a performant manner.

Examples

For further reference, do have a look at the basic modules present in modularization.*Module.java.