Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-46325][CONNECT] Remove unnecessary override functions when constructing WrappedCloseableIterator in ResponseValidator#wrapIterator #44255

Closed
wants to merge 1 commit into from

Conversation

LuciferYang
Copy link
Contributor

What changes were proposed in this pull request?

This pr removes the overridden hasNext and close functions in the construction of WrappedCloseableIterator in ResponseValidator#wrapIterator, as these functions are identical to those defined in WrappedCloseableIterator.

  • WrappedCloseableIterator

private[sql] abstract class WrappedCloseableIterator[E] extends CloseableIterator[E] {
def innerIterator: Iterator[E]
override def next(): E = innerIterator.next()
override def hasNext: Boolean = innerIterator.hasNext
override def close(): Unit = innerIterator match {
case it: CloseableIterator[E] => it.close()
case _ => // nothing
}
}

  • ResponseValidator#wrapIterator

def wrapIterator[T <: GeneratedMessageV3, V <: CloseableIterator[T]](
inner: V): WrappedCloseableIterator[T] = {
new WrappedCloseableIterator[T] {
override def innerIterator: Iterator[T] = inner
override def hasNext: Boolean = {
innerIterator.hasNext
}
override def next(): T = {
verifyResponse {
innerIterator.next()
}
}
override def close(): Unit = {
innerIterator match {
case it: CloseableIterator[T] => it.close()
case _ => // nothing
}
}
}
}

Why are the changes needed?

Remove unnecessary override functions.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

dbatomic pushed a commit to dbatomic/spark that referenced this pull request Dec 11, 2023
…structing `WrappedCloseableIterator` in `ResponseValidator#wrapIterator`

### What changes were proposed in this pull request?
This pr removes the overridden `hasNext` and `close` functions in the construction of `WrappedCloseableIterator` in `ResponseValidator#wrapIterator`, as these functions are identical to those defined in `WrappedCloseableIterator`.

- WrappedCloseableIterator

https://github.com/apache/spark/blob/9ffdcc398ed5560f34778d005da697f6ad0a15ee/connector/connect/common/src/main/scala/org/apache/spark/sql/connect/client/CloseableIterator.scala#L30-L42

- ResponseValidator#wrapIterator

https://github.com/apache/spark/blob/9ffdcc398ed5560f34778d005da697f6ad0a15ee/connector/connect/common/src/main/scala/org/apache/spark/sql/connect/client/ResponseValidator.scala#L62-L85

### Why are the changes needed?
Remove unnecessary override functions.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#44255 from LuciferYang/SPARK-46325.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants