New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix KQL and PRQL distributed tables queries #59674
Fix KQL and PRQL distributed tables queries #59674
Conversation
This is an automated comment for commit e23dd5b with description of existing statuses. It's updated for the latest CI running ❌ Click here to open a full report in a separate page Successful checks
|
src/Client/HedgedConnections.cpp
Outdated
@@ -177,6 +177,10 @@ void HedgedConnections::sendQuery( | |||
{ | |||
Settings modified_settings = settings; | |||
|
|||
// Kusto queries to replicas are transformed to ClickHouse-SQL. Change the setting before sending. | |||
if (modified_settings.dialect == Dialect::kusto) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks wrong. Why only Kusto?
What about the PRQL dialect?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will investigate. I hadn't seen any issues submitted for PRQL having the same issue with distributed tables, and thought maybe the dialect was doing something else making it unaffected. I will report what I find.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can confirm that this issue also effects PRQL. Ex. error output
Received exception from server (version 23.12.2):
Code: 62. DB::Exception: Received from localhost:9000. DB::Exception: Received from clickhouse03:9000. DB::Exception: PRQL syntax error: 'Error:
╭─[:1:1]
│
1 │ SELECT `shared_test_table`.`id` FROM `default`.`shared_test_table`
│ ───┬──
│ ╰──── Unknown name
───╯
'. (SYNTAX_ERROR)
Here, we can see the same behavior, of the transformed SQL statement trying to be executed in the incorrect dialect.
I will extend my change and add appropriate tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing tests.
Also, write a test with PRQL. |
tests/queries/0_stateless/02985_dialects_with_distributed_tables.sql
Outdated
Show resolved
Hide resolved
@@ -126,6 +126,10 @@ void MultiplexedConnections::sendQuery( | |||
|
|||
Settings modified_settings = settings; | |||
|
|||
// Kusto and PRQL queries to replicas are transformed to ClickHouse-SQL. Ensure the setting before sending. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code is non-generic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had the change that was made in 01e203d in an earlier commit, and ultimately changed it to what I had because of backwards compatibility tests throwing a Unknown setting dialect
.
I definitely agree that the generic form of blanket setting the dialect is the better way to go.
src/Client/HedgedConnections.cpp
Outdated
@@ -177,6 +177,10 @@ void HedgedConnections::sendQuery( | |||
{ | |||
Settings modified_settings = settings; | |||
|
|||
// Kusto and PRQL queries to replicas are transformed to ClickHouse-SQL. Ensure the setting before sending. | |||
if (modified_settings.dialect == Dialect::kusto || modified_settings.dialect == Dialect::prql) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've simplified the code.
d08ac82
to
e23dd5b
Compare
Sorry, editing through web UI was a bad idea. Continued here: #60470 |
…buted-tables Merging #59674.
Done. |
…kql-distributed-tables Merging ClickHouse#59674.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fixed a minor bug that prevented distributed table queries sent from either KQL or PRQL dialect clients to be executed on replicas.
Documentation entry for user-facing changes
This PR address #56289 and #57378 which found that queries using a distributed table with the kusto dialect resulted in a syntax error.
Root cause of this issue seems to be with how the queries are passed onto replicas for execution. A query such as
is transformed to
When executing this, the query is sent to connected replicas, which includes a modified copy of the settings. Within this, was the inclusion of the dialect change, which caused the replica to try and execute an SQL query with the KQL parser. Modifying the dialect prior to distribution resolves this issue.
This PR depends on #59626 being merged prior.
Edit: As observed below, this bug also extends to PRQL for the same reason. Adjustments have been made to just blanket set the dialect to ClickHouse-SQL before querying replicas.