Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CALCITE-2322] Add fetch size support to connection url and JDBC state… #49

Conversation

Projects
None yet
3 participants
@kminder
Copy link

kminder commented May 23, 2018

Adds the support for both a URL connection param fetch_size and the JDBD Statement API setFetchSize. This can be used to tune performance for queries that return a large number of rows.

@laurentgo
Copy link
Contributor

laurentgo left a comment

I looked at a couple of jdbc driver, and looks like connection property is named defaultFetchSize or equivalent, because this is the value to be used when Statement/ResultSet fetchSize is set to 0.

Currently, the code doesn't check that the value set in fetchSize is 0 or greater. Also if value is 0, I don't think the value is then replaced by the default (same for AvaticaResultSet). The reason is probably because it's not possible to change the fetch size for the current current cursor, that said it would be nice to clean up a bit AvaticaStatement and AvaticaResult (and ConnectionConfig) to validate (default) fetch size value, and in case 0 is used to replace it by the default value on the fly (and not just at construction time)

@@ -108,6 +109,7 @@ protected AvaticaStatement(AvaticaConnection connection,
this.resultSetType = resultSetType;
this.resultSetConcurrency = resultSetConcurrency;
this.resultSetHoldability = resultSetHoldability;
this.fetchSize = connection.config().fetchSize(); // Default to connection config fetch size.

This comment has been minimized.

Copy link
@laurentgo

laurentgo May 30, 2018

Contributor

let's remove the default value set for the field declaration

This comment has been minimized.

Copy link
@kminder

kminder Jun 1, 2018

Author

I'm not clear on what you are suggesting here. Are you suggesting this be defaulted to 0 and have the real value resolved in getFetchSize()? I went with the approach I did because a setting made via the connection url is accessed through the connection object. The connection reference isn't stored within AvaticaStatement so the connection value needs to be propagated down somehow. You are correct that if the value is explicitly set to 0 either via url param or via Statement.setFetchSize or ResultSet.setFetchSize I have no idea what will happen. All I can say is that I didn't change that behavior with this change. It will do whatever is would have done before.

This comment has been minimized.

Copy link
@laurentgo

laurentgo Jun 7, 2018

Contributor

the value is both set in the field declaration and in the constructor, just suggesting to remove the default value in the field initialization...

HostnameVerification.class, false),

/** Fetch size limit, default is 100 rows. */
FETCH_SIZE("fetch_size", Type.NUMBER, AvaticaStatement.DEFAULT_FETCH_SIZE, false);

This comment has been minimized.

Copy link
@laurentgo

laurentgo May 30, 2018

Contributor

I would recommend to name it DEFAULT_FETCH_SIZE

This comment has been minimized.

Copy link
@kminder

kminder Jun 1, 2018

Author

I'm not clear on your suggestion. Are you suggesting that the url connection parameter be named "DEFAULT_FETCH_SIZE" or as you suggest above "defaultFetchSize". You mention having looked at a few other drivers I've looked too and don't see much consistency. I chose "fetch_size" because that seemed consistent with the other parameters. Would "default_fetch_size" be agreeable?

This comment has been minimized.

Copy link
@laurentgo

laurentgo Jun 7, 2018

Contributor

yes, default_fetch_size for the jdbc property, which means the enum become DEFAULT_FETCH_SIZE (that was my point :) )

@F21

This comment has been minimized.

Copy link
Member

F21 commented Jun 12, 2018

@kminder Do you think it would be possible to get this change in by the end of this week?

@asfgit asfgit force-pushed the apache:master branch from 2a80eec to 146d310 Sep 23, 2018

@F21

This comment has been minimized.

Copy link
Member

F21 commented Nov 13, 2018

@kminder We are planning to release Avatica 1.13.0 soon and would love to get this in for the release. Would you be able to take another look at this PR?

@asfgit asfgit force-pushed the apache:master branch 5 times, most recently from ff37d89 to a8617e0 Nov 16, 2018

@risdenk risdenk changed the title CALCITE-2322: Add fetch size support to connection url and JDBC state… [CALCITE-2322] Add fetch size support to connection url and JDBC state… Feb 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.