Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support JDBC explicitPrepare flag #117

Merged
merged 1 commit into from
Nov 17, 2023
Merged

Conversation

lpoulain
Copy link
Collaborator

@lpoulain lpoulain commented Nov 2, 2023

Now that JDBC has a new explicitPrepare flag (Trino 431+) to use EXECUTE IMMEDIATE for prepared statements (Trino 418+), allow a new database option to use this (it is set on by default). This is achieved by a few changes:

  • In metabase-plugin.yaml the new option is added
  • In connectivity.clj the JDBC parameters are modified accordingly
  • In execute.clj a PreparedStatement proxy is returned. This is because Metabase calls getParameterMetaData() which would negate the upsides of EXECUTE IMMEDIATE. This is used by Metabase just to count the number of arguments to issue a message. Not defining getParameterMetaData() in the proxy is not an issue because 1) Metabase has a try/catch when calling this method and 2) we issue the exact same error message, except we do it after the query is executed
  • Two custom error messages are introduced in case customers are using a version of Trino older than 418. The first one when defining a connection (starburst.clj), where we know that if we get a Expecting: 'USING' error when testing the connection it's because of an incompatible Trino version. The second one when running a query (e.g. the customer downgrades Trino after setting up the connection) in which case this may be the issue but we cannot be certain (handle-execution-error method in execute.clj)

As far as testing goes, we are running the whole Metabase test suite against both options (hence a modification in Makefile). The downside is that we are running many tests which are independent of the explicitPrepare option.

New flag when defining a connection (including error message):

Screenshot 2023-10-26 at 3 42 22 PM

Error message when a query fails with Expecting: 'USING':

Screenshot 2023-10-26 at 11 58 56 AM

(setDate
([index val] (.setDate stmt index val))
([index val cal] (.setDate stmt index val cal)))
(close [] (.close stmt)))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the only methods invoked by Metabase?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the only ones I have found (save getParameterMetaData), both by running all the tests and looking at the Metabase source code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added several other methods. Not sure if you want more.

Copy link
Member

@andrewdibiasio6 andrewdibiasio6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big change for clients, please update change log. This will require a major version bump as well.

Copy link
Member

@andrewdibiasio6 andrewdibiasio6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments

Makefile Outdated Show resolved Hide resolved
@lpoulain
Copy link
Collaborator Author

lpoulain commented Nov 7, 2023

@andrewdibiasio6 changelog updated

type: boolean
required: false
default: false
description: Requires Starburst Galaxy, Starburst Enterprise (version 420+), or Trino (version 418+)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Starburst Enterprise (version 420-e or higher)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

; - This message is caught by the driver and replaced with the exact same Metabase message
; In the end, the exact same message is presented to the user when the number of arguments is
; incorrect except we now execute the query to display the error message
(let [ps (proxy [java.sql.PreparedStatement] []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using this conditionally, only when optimized prepared statements are enabled?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the PR to use the old code when TrinoConnection.useExplicitPrepare() returns true and the new code otherwise.

Copy link
Member

@andrewdibiasio6 andrewdibiasio6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comments

CHANGELOG.md Show resolved Hide resolved
Makefile Show resolved Hide resolved
Support JDBC explicitPrepare flag
@@ -284,6 +284,57 @@
(is (= (str "impersonate:true")
(:clientInfo jdbc-spec))))))

(defn prepared-statements-helper
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Since we have this test that tests prepared-optimized, not sure we need the make optimized make command.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added just in case should we suspect some issue with this flag (e.g. the standard Metabase tests using the prepared statements a slightly different way)

@lpoulain lpoulain merged commit b603b14 into main Nov 17, 2023
2 checks passed
@lpoulain lpoulain linked an issue Nov 20, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Trino's EXECUTE IMMEDIATE
3 participants