[SPARK-37268][SQL] Remove unused method call in FileScanRDD #34545
Conversation
HyukjinKwon
changed the title
| @@ -107,7 +107,6 @@ class FileScanRDD( | |||
| val nextElement = currentIterator.next() | |||
| // TODO: we should have a better separation of row based and batch based scan, so that we | |||
| // don't need to run this `if` for every record. | |||
| val preNumRecordsRead = inputMetrics.recordsRead | |||
There was a problem hiding this comment.
Looks fine but it would have been better which PR removed the last usage of this in order to make any mistake.
There was a problem hiding this comment.
Changed from SPARK-25674.
This PR directly uses inputMetrics.recordsRead instead of using preNumRecordsRead
|
@zuston please fill the PR description properly. |
Done. @HyukjinKwon |
|
ok to test |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Test build #145107 has finished for PR 34545 at commit
|
There was a problem hiding this comment.
+1, LGTM. Thank you, @zuston and @HyukjinKwon . I also confirmed that SPARK-25647 remove it. Merged to master for Apache Spark 3.3.
|
Welcome to the Apache Spark community, @zuston . |
Glad to join Spark community. Thanks @dongjoon-hyun @HyukjinKwon. |
What changes were proposed in this pull request?
Remove unused method call in FileScanRDD
Why are the changes needed?
In PR of SPARK-25674, it directly uses inputMetrics.recordsRead instead of using preNumRecordsRead. So it's necessary to remove unused method call.
Does this PR introduce any user-facing change?
No
How was this patch tested?
No.