Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix getting views for Hive 2.3+ metastore #833

Conversation

3 participants
@findepi
Copy link
Member

commented May 28, 2019

Hive 2.3 metastore provides more space for table parameter values. On
certain databases (e.g. Derby, Oracle) it uses CLOB and these databases
disallow = predicates over CLOB values. At the same time, they allow
LIKE predicates over them.

This fixes SHOW TABLES and queries over information_schema.tables.

Also, fixes prestodb/presto#10735, nico-arianto/big-data-local#1.

@cla-bot cla-bot bot added the cla-signed label May 28, 2019

@electrum

This comment has been minimized.

Copy link
Member

commented May 28, 2019

Nice trick!

@findepi

This comment has been minimized.

Copy link
Member Author

commented May 28, 2019

@electrum see https://issues.apache.org/jira/browse/HIVE-12274 ("Increase width of columns used for general configuration in the metastore.") diff, how it touches the tests.

argh, but Hive < 2.3 did not support LIKE (hence lots of product tests failures).

@findepi findepi force-pushed the findepi:findepi/master/fix-getting-views-for-hive-2-3-metastore-7f951e branch from 8e9448d to c5a8354 May 29, 2019

@findepi

This comment has been minimized.

Copy link
Member Author

commented May 29, 2019

@martint @electrum please have a look, I took a "feature detection" approach

Fix getting views for Hive 2.3+ metastore
Hive 2.3 metastore provides more space for table parameter values. On
certain databases (e.g. Derby, Oracle) it uses CLOB and these databases
disallow `=` predicates over CLOB values. At the same time, they allow
`LIKE` predicates over them.

This fixes `SHOW TABLES` and queries over `information_schema.tables`.

@findepi findepi force-pushed the findepi:findepi/master/fix-getting-views-for-hive-2-3-metastore-7f951e branch from c5a8354 to d5b6f27 May 30, 2019

@electrum electrum requested a review from martint May 31, 2019

@@ -117,6 +117,9 @@
private final Duration maxRetryTime;
private final int maxRetries;

private volatile boolean metastoreKnownToSupportTableParamEqualsPredicate;

This comment has been minimized.

Copy link
@electrum

electrum May 31, 2019

Member

Maybe shorten KnownToSupport to Supports

This comment has been minimized.

Copy link
@findepi

findepi May 31, 2019

Author Member

Then i wouldn't be able to use false as an initial value (because initially i don't know yet).

return views;
}
catch (TException | RuntimeException secondException) {
if (firstException != secondException) {

This comment has been minimized.

Copy link
@martint

martint May 31, 2019

Member

Why would firstException == secondException?

This comment has been minimized.

Copy link
@findepi

findepi May 31, 2019

Author Member

Technically you can keep an exception and rethrow it multiple times. I see no particular reason why this could happen here.
Apparently we often guard calls to java.lang.Throwable#addSuppressed like this (for apparently no particular reason), so I am blindly following an established convention 😉

This comment has been minimized.

Copy link
@electrum

electrum May 31, 2019

Member

It seems unlikely to happen in practice but is a good safety measure.

@findepi findepi merged commit 179ffe4 into prestosql:master Jun 4, 2019

2 checks passed

Travis CI - Pull Request Build Passed
Details
verification/cla-signed
Details

@findepi findepi deleted the findepi:findepi/master/fix-getting-views-for-hive-2-3-metastore-7f951e branch Jun 4, 2019

@findepi findepi referenced this pull request Jun 4, 2019

Closed

Release notes for 314 #879

2 of 6 tasks complete

@findepi findepi added this to the 314 milestone Jun 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.