New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added SparkSQL support based on HiveDialect #187
base: master
Are you sure you want to change the base?
Conversation
This fixes #150 without breaking the original Hive engine. |
Any progress on this? |
+1 |
+1 to get this PR merged :) |
Do you thought also about changing get_view_names, get_table_names result based on isTemporary column ? |
bump |
@tanhaoqin this is failing the tests for some reason, maybe rebasing would fix the issue? |
f0e8692
to
cd6e284
Compare
SparkSQL is made to be almost compatible with HiveQL. However, the `show tables` syntax for Spark SQL is slightly different. As such, a new SparkSQL dialect is created based on the original HiveDialect with `get_table_names` modified to accommodate the results returned from Spark SQL's 'SHOW TABLES'.
Changed some stuff so there's no longer any conflicts. I am no longer working on the project that required this feature though. Didn't have time to write tests as I am not using any Hive/Presto on my side so the original Hive/Presto tests will fail anyway. |
Many tests done by |
|
SparkSQL is made to be almost compatible with HiveQL. However, the
show tables
syntax for Spark SQL is slightly different. As such, a newSparkSQL dialect is created based on the original HiveDialect with
get_table_names
modified to accommodate the results returned from Spark SQL's'SHOW TABLES'.