This repository has been archived by the owner on May 12, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 152
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
2e47b4c
commit 79d40d5
Showing
1 changed file
with
49 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
Filter | ||
============= | ||
|
||
## Operator Objective | ||
This operator receives an POJO ([Plain Old Java Object](https://en.wikipedia.org/wiki/Plain_Old_Java_Object)) as an incoming tuple | ||
and based on the filter condition it shall emit the filtered POJO on one output port and the rest on another output port. | ||
|
||
Filter operator supports quasi Java expressions to specify filter rule. | ||
|
||
Filter operator does not hold any state and is **idempotent**, **fault-tolerant** and **statically/dynamically partitionable**. | ||
|
||
## Operator Usecase | ||
1. ***Customer data*** usually contains a field customer category/segment. One wants some analysis to be done for specific customer segment. One could filter the records based on segment. | ||
2. ***Log data*** processing pipeline may want to filter logs from specific machine/router/switch. | ||
|
||
## Operator Information | ||
1. Operator location: ***malhar-library*** | ||
2. Available since: ***3.5.0*** | ||
3. Operator state: ***Evolving*** | ||
3. Java Packages: | ||
* Operator: ***[com.datatorrent.lib.filter.FilterOperator](https://www.datatorrent.com/docs/apidocs/com/datatorrent/lib/filter/FilterOperator.html)*** | ||
|
||
## Properties, Attributes and Ports | ||
### <a name="props"></a>Properties of FilterOperator | ||
| **Property** | **Description** | **Type** | **Mandatory** | **Default Value** | | ||
| -------- | ----------- | ---- | ------------------ | ------------- | | ||
| *condition* | condition/expression with which Filtering is done. | String | Yes | N/A | | ||
| *additionalExpressionFunctions* | List of import classes/method that should be made statically available to expression to use. | `List< String>`| No | Empty List | | ||
|
||
### Platform Attributes that influences operator behavior | ||
| **Attribute** | **Description** | **Type** | **Mandatory** | | ||
| -------- | ----------- | ---- | ------------------ | | ||
| *port.input.attr.TUPLE_CLASS* | TUPLE_CLASS attribute on input port indicates the class of POJO which incoming tuple | Class or FQCN| Yes | | ||
|
||
|
||
### Ports | ||
| **Port** | **Description** | **Type** | **Mandatory** | | ||
| -------- | ----------- | ---- | ------------------ | | ||
| *input* | Tuple which needs to be filtered are received on this port | Object (POJO) | Yes | | ||
| *truePort* | Tuples which satisfies `condition` are emitted on this port | Object (POJO) | No | | ||
| *falsePort* | Tuples which does not satisfy `condition` are emitted on this port | Object (POJO) | No | | ||
|
||
## Limitations | ||
Current `FilterOperator` has following limitation: | ||
|
||
1. [APEXMALHAR-2175](https://issues.apache.org/jira/browse/APEXMALHAR-2175) : Filter condition is not able to correctly handle java reserved words. | ||
|
||
## Example | ||
Example for `FilterOperator` can be found at: [https://github.com/DataTorrent/examples/tree/master/tutorials/filter](https://github.com/DataTorrent/examples/tree/master/tutorials/filter) |