Skip to content

Commit

Permalink
[SPARK-38391][SPARK-38768][SQL][FOLLOWUP] Add comments for `pushLimit…
Browse files Browse the repository at this point in the history
…` and `pushTopN` of `PushDownUtils`

### What changes were proposed in this pull request?
`pushLimit` and `pushTopN` of `PushDownUtils` returns tuple of boolean. It will be good to explain what the boolean value represents.

### Why are the changes needed?
Make DS V2 API more friendly to developers.

### Does this PR introduce _any_ user-facing change?
'No'.
Just update comments.

### How was this patch tested?
N/A

Closes #36092 from beliefer/SPARK-38391_SPARK-38768_followup.

Authored-by: Jiaan Geng <beliefer@163.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
beliefer authored and dongjoon-hyun committed Apr 11, 2022
1 parent bf19a5e commit c4397cb
Showing 1 changed file with 10 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,11 @@ object PushDownUtils extends PredicateHelper {
}

/**
* Pushes down LIMIT to the data source Scan
* Pushes down LIMIT to the data source Scan.
*
* @return the tuple of Boolean. The first Boolean value represents whether to push down, and
* the second Boolean value represents whether to push down partially, which means
* Spark will keep the Limit and do it again.
*/
def pushLimit(scanBuilder: ScanBuilder, limit: Int): (Boolean, Boolean) = {
scanBuilder match {
Expand All @@ -127,7 +131,11 @@ object PushDownUtils extends PredicateHelper {
}

/**
* Pushes down top N to the data source Scan
* Pushes down top N to the data source Scan.
*
* @return the tuple of Boolean. The first Boolean value represents whether to push down, and
* the second Boolean value represents whether to push down partially, which means
* Spark will keep the Sort and Limit and do it again.
*/
def pushTopN(
scanBuilder: ScanBuilder,
Expand Down

0 comments on commit c4397cb

Please sign in to comment.