Skip to content

Commit

Permalink
update the comments.
Browse files Browse the repository at this point in the history
  • Loading branch information
gatorsmile committed Dec 9, 2016
1 parent b60f7bb commit 04b0e9c
Showing 1 changed file with 2 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -168,12 +168,11 @@ object ExtractPythonUDFs extends Rule[SparkPlan] with PredicateHelper {
}
}

// Split the original FilterExec to two FilterExecs. The upper FilterExec only contains
// Python UDF and non-deterministic predicates.
// Split the original FilterExec to two FilterExecs. Only push down the first few predicates
// that are all deterministic.
private def trySplitFilter(plan: SparkPlan): SparkPlan = {
plan match {
case filter: FilterExec =>
// Only push down the first few predicates that are all deterministic
val (candidates, containingNonDeterministic) =
splitConjunctivePredicates(filter.condition).span(_.deterministic)
val (pushDown, rest) = candidates.partition(!hasPythonUDF(_))
Expand Down

0 comments on commit 04b0e9c

Please sign in to comment.