Iterator predicate pushdown

Custom StorageHandler implementations can use the Hive API to analyze one or more predicates appearing in query WHERE clauses for direct. The idea is that custom Serde used in the storage handler is better equipped to incorporate the predicate(s) into query restrictions for the underlying datastore, thus avoiding the need to Hive filter at higher level.

The AccumuloStorageHandler supports decomposition of WHERE clause predicates involving <, >, <=, >=, !=, and = over any of the defined Hive columns, including the rowID. Predicate conditions involving these operators are converted to one or more IteraterSettings that filter rows directly from the TabletServers.

The Hive API currently only allows predicate decomposition to the storage handler based for WHERE clauses that consist of purely conjuctive predicates.

For example:

SELECT * FROM table1 WHERE foo = 'hi' AND bar = 'there';

The above query consists of two predicates (foo = 'hi') and ('bar' = 'there'). Because they are combined into a single conjunctive restriction, they both are available to the StorageHandler.

In the case of SELECT * FROM table1 WHERE foo = 'hi' OR bar = 'there'; Neither of the predicates are available to the storage handler.

If even a singe disjunctive condition is found by the Hive predicate analyzer, the entire clause is evaluated at the Hive filter-level, and is not available for direct storage handler decomposition. So in the case of SELECT * FROM table1 WHERE foo = 'hi' AND bar = 'there' OR 'foo2' = 'whoa';, despite having two predicates ANDed together, the presence of the final OR operator means nothing can be passed down. This is a currently limitation with the Hive API.

Resultantly, we can naively assume predicates sent to the Accumulo storage handler are meant to be ANDed together else they wouldn't be visible to us in the first place.

Predicates received by the storage handler are mapped 1-to-1 with a filter IteratorSetting. The Compare API in the AccumuloStorageHandler makes use of a single class PrimitiveCompareFilter that extends WholeRowFilteringIterator.

As part of the Compare API, each IteratorSetting is supplied as part of the configuration options 5 critical parameters necessary to carry down the Hive column filtering.

1. The column family and qualifier corresponding to the Hive column in question.

add details of range restrictions.