Threading

Loghub uses threads transparently to do parallel processing and don’t lock. A number of workers threads are created using the numWorkers property. Each thread take a waiting event for a pipeline, processes it through all the processing steps, included the sub-pipeline, and at the end it will be sent to wait again for processing by the next pipeline or an output.

Input

Input object are mono threaded as usually they listen on a single network port that will be difficult to share. The developer must then take care of blocking code.

Output

Processors

Each processor step in a pipeline exists as a single instance that will be shared by all the workers thread. If a processor class is used in different pipeline steps, they will be different instances.

As a processor is generally stateless, that should not be a problem. But if some non thread safe object are to be used, extra care must be used, a java.lang.ThreadLocal is a very efficient solution to this issue.