-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-54014][CONNECT] Support max rows for SparkConnectStatement #52742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
84093f3 to
a669c75
Compare
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Outdated
Show resolved
Hide resolved
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Outdated
Show resolved
Hide resolved
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Outdated
Show resolved
Hide resolved
1116ffc to
3540af6
Compare
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Outdated
Show resolved
Hide resolved
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Outdated
Show resolved
Hide resolved
...ent/jdbc/src/main/scala/org/apache/spark/sql/connect/client/jdbc/SparkConnectStatement.scala
Show resolved
Hide resolved
|
rebase on master to resolve Python protobuf version mismatch issue (no code change) |
### What changes were proposed in this pull request?
This PR implements the two methods of the `java.sql.Statement` interface for `SparkConnectStatement`
```
/**
* Retrieves the maximum number of rows that a
* {code ResultSet} object produced by this
* {code Statement} object can contain. If this limit is exceeded,
* the excess rows are silently dropped.
*
* return the current maximum number of rows for a {code ResultSet}
* object produced by this {code Statement} object;
* zero means there is no limit
* throws SQLException if a database access error occurs or
* this method is called on a closed {code Statement}
* see #setMaxRows
*/
int getMaxRows() throws SQLException;
/**
* Sets the limit for the maximum number of rows that any
* {code ResultSet} object generated by this {code Statement}
* object can contain to the given number.
* If the limit is exceeded, the excess
* rows are silently dropped.
*
* param max the new max rows limit; zero means there is no limit
* throws SQLException if a database access error occurs,
* this method is called on a closed {code Statement}
* or the condition {code max >= 0} is not satisfied
* see #getMaxRows
*/
void setMaxRows(int max) throws SQLException;
```
### Why are the changes needed?
Implement more JDBC APIs.
### Does this PR introduce _any_ user-facing change?
No, it's new feature.
### How was this patch tested?
New UTs are added.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52742 from pan3793/SPARK-54014.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 07cab00)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
|
Merged to master. Thank you, @pan3793 and @LuciferYang . |
|
I'm planning to backport this pr to branch-4.1. Additionally, to make this feature basically available in Spark 4.1, there are another 9 pending tickets that I hope can be completed and merged into branch-4.1 by November 15th: https://issues.apache.org/jira/browse/SPARK-53484
If you have any other suggestions, please let me know. @dongjoon-hyun Thanks ~ |
|
Oh, I backported this already to And, yes, of course, you can backport all late arrival patches until November 15th. So, feel free to proceed as the member of Apache Spark PMC. |
|
@dongjoon-hyun @LuciferYang, many thanks for your help in advancing this new feature. |
|
Thank you for your clarification and support. @dongjoon-hyun |
### What changes were proposed in this pull request?
This PR implements the two methods of the `java.sql.Statement` interface for `SparkConnectStatement`
```
/**
* Retrieves the maximum number of rows that a
* {code ResultSet} object produced by this
* {code Statement} object can contain. If this limit is exceeded,
* the excess rows are silently dropped.
*
* return the current maximum number of rows for a {code ResultSet}
* object produced by this {code Statement} object;
* zero means there is no limit
* throws SQLException if a database access error occurs or
* this method is called on a closed {code Statement}
* see #setMaxRows
*/
int getMaxRows() throws SQLException;
/**
* Sets the limit for the maximum number of rows that any
* {code ResultSet} object generated by this {code Statement}
* object can contain to the given number.
* If the limit is exceeded, the excess
* rows are silently dropped.
*
* param max the new max rows limit; zero means there is no limit
* throws SQLException if a database access error occurs,
* this method is called on a closed {code Statement}
* or the condition {code max >= 0} is not satisfied
* see #getMaxRows
*/
void setMaxRows(int max) throws SQLException;
```
### Why are the changes needed?
Implement more JDBC APIs.
### Does this PR introduce _any_ user-facing change?
No, it's new feature.
### How was this patch tested?
New UTs are added.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes apache#52742 from pan3793/SPARK-54014.
Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

What changes were proposed in this pull request?
This PR implements the two methods of the
java.sql.Statementinterface forSparkConnectStatementWhy are the changes needed?
Implement more JDBC APIs.
Does this PR introduce any user-facing change?
No, it's new feature.
How was this patch tested?
New UTs are added.
Was this patch authored or co-authored using generative AI tooling?
No.