Skip to content

Conversation

@pan3793
Copy link
Member

@pan3793 pan3793 commented Oct 27, 2025

What changes were proposed in this pull request?

This PR implements the two methods of the java.sql.Statement interface for SparkConnectStatement

    /**
     * Retrieves the maximum number of rows that a
     * {@code ResultSet} object produced by this
     * {@code Statement} object can contain.  If this limit is exceeded,
     * the excess rows are silently dropped.
     *
     * @return the current maximum number of rows for a {@code ResultSet}
     *         object produced by this {@code Statement} object;
     *         zero means there is no limit
     * @throws SQLException if a database access error occurs or
     * this method is called on a closed {@code Statement}
     * @see #setMaxRows
     */
    int getMaxRows() throws SQLException;

    /**
     * Sets the limit for the maximum number of rows that any
     * {@code ResultSet} object  generated by this {@code Statement}
     * object can contain to the given number.
     * If the limit is exceeded, the excess
     * rows are silently dropped.
     *
     * @param max the new max rows limit; zero means there is no limit
     * @throws SQLException if a database access error occurs,
     * this method is called on a closed {@code Statement}
     *            or the condition {@code max >= 0} is not satisfied
     * @see #getMaxRows
     */
    void setMaxRows(int max) throws SQLException;

Why are the changes needed?

Implement more JDBC APIs.

Does this PR introduce any user-facing change?

No, it's new feature.

How was this patch tested?

New UTs are added.

Was this patch authored or co-authored using generative AI tooling?

No.

@pan3793 pan3793 force-pushed the SPARK-54014 branch 2 times, most recently from 84093f3 to a669c75 Compare October 27, 2025 12:59
@pan3793
Copy link
Member Author

pan3793 commented Oct 27, 2025

cc @LuciferYang @dongjoon-hyun

@pan3793 pan3793 force-pushed the SPARK-54014 branch 2 times, most recently from 1116ffc to 3540af6 Compare November 3, 2025 08:37
@pan3793
Copy link
Member Author

pan3793 commented Nov 5, 2025

rebase on master to resolve Python protobuf version mismatch issue (no code change)

dongjoon-hyun pushed a commit that referenced this pull request Nov 5, 2025
### What changes were proposed in this pull request?

This PR implements the two methods of the `java.sql.Statement` interface for `SparkConnectStatement`
```
    /**
     * Retrieves the maximum number of rows that a
     * {code ResultSet} object produced by this
     * {code Statement} object can contain.  If this limit is exceeded,
     * the excess rows are silently dropped.
     *
     * return the current maximum number of rows for a {code ResultSet}
     *         object produced by this {code Statement} object;
     *         zero means there is no limit
     * throws SQLException if a database access error occurs or
     * this method is called on a closed {code Statement}
     * see #setMaxRows
     */
    int getMaxRows() throws SQLException;

    /**
     * Sets the limit for the maximum number of rows that any
     * {code ResultSet} object  generated by this {code Statement}
     * object can contain to the given number.
     * If the limit is exceeded, the excess
     * rows are silently dropped.
     *
     * param max the new max rows limit; zero means there is no limit
     * throws SQLException if a database access error occurs,
     * this method is called on a closed {code Statement}
     *            or the condition {code max >= 0} is not satisfied
     * see #getMaxRows
     */
    void setMaxRows(int max) throws SQLException;
```

### Why are the changes needed?

Implement more JDBC APIs.

### Does this PR introduce _any_ user-facing change?

No, it's new feature.

### How was this patch tested?

New UTs are added.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #52742 from pan3793/SPARK-54014.

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 07cab00)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun
Copy link
Member

Merged to master. Thank you, @pan3793 and @LuciferYang .

@LuciferYang
Copy link
Contributor

LuciferYang commented Nov 6, 2025

I'm planning to backport this pr to branch-4.1. Additionally, to make this feature basically available in Spark 4.1, there are another 9 pending tickets that I hope can be completed and merged into branch-4.1 by November 15th: https://issues.apache.org/jira/browse/SPARK-53484

image

If you have any other suggestions, please let me know. @dongjoon-hyun Thanks ~

@dongjoon-hyun
Copy link
Member

Oh, I backported this already to branch-4.1 when I merged this. Sorry for making you confused by my previous wrong comment, @LuciferYang . Here is the commit.

And, yes, of course, you can backport all late arrival patches until November 15th. So, feel free to proceed as the member of Apache Spark PMC.

@pan3793
Copy link
Member Author

pan3793 commented Nov 6, 2025

@dongjoon-hyun @LuciferYang, many thanks for your help in advancing this new feature.

@LuciferYang
Copy link
Contributor

Thank you for your clarification and support. @dongjoon-hyun

huangxiaopingRD pushed a commit to huangxiaopingRD/spark that referenced this pull request Nov 25, 2025
### What changes were proposed in this pull request?

This PR implements the two methods of the `java.sql.Statement` interface for `SparkConnectStatement`
```
    /**
     * Retrieves the maximum number of rows that a
     * {code ResultSet} object produced by this
     * {code Statement} object can contain.  If this limit is exceeded,
     * the excess rows are silently dropped.
     *
     * return the current maximum number of rows for a {code ResultSet}
     *         object produced by this {code Statement} object;
     *         zero means there is no limit
     * throws SQLException if a database access error occurs or
     * this method is called on a closed {code Statement}
     * see #setMaxRows
     */
    int getMaxRows() throws SQLException;

    /**
     * Sets the limit for the maximum number of rows that any
     * {code ResultSet} object  generated by this {code Statement}
     * object can contain to the given number.
     * If the limit is exceeded, the excess
     * rows are silently dropped.
     *
     * param max the new max rows limit; zero means there is no limit
     * throws SQLException if a database access error occurs,
     * this method is called on a closed {code Statement}
     *            or the condition {code max >= 0} is not satisfied
     * see #getMaxRows
     */
    void setMaxRows(int max) throws SQLException;
```

### Why are the changes needed?

Implement more JDBC APIs.

### Does this PR introduce _any_ user-facing change?

No, it's new feature.

### How was this patch tested?

New UTs are added.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#52742 from pan3793/SPARK-54014.

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants