Skip to content

Conversation

@berkaysynnada
Copy link
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

While we can write CLI commands sequentially such as:

CREATE TABLE test1 (
    c1 VARCHAR(255),
    c2 INT,
    c3 INT
);

INSERT INTO test1 (c1, c2, c3) VALUES ('c', 2, 1);

SELECT * FROM test1;

CLI gives error to write more commands after "CREATE EXTERNAL TABLE":

CREATE EXTERNAL TABLE test1 (
    c1 VARCHAR(255),
    c2 INT,
    c3 INT,
    ...
)
STORED AS CSV
WITH HEADER ROW
LOCATION '/Users/berkaysahin/Desktop/datafusion-upstream/testing/data/csv/aggregate_test_100.csv';

SELECT * FROM test1;

🤔 Invalid statement: sql parser error: Expected end of statement, found: SELECT

What changes are included in this PR?

This PR addresses the issue that we needed to enter commands one by one for creating external tables. It resolves this by splitting the entire string at semicolons, ensuring each segment passes parser checks one by one, and then executing them sequentially.

Are these changes tested?

Are there any user-facing changes?

Copy link
Contributor

@mustafasrepo mustafasrepo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!. I wonder though how we can test this support in the code base.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a nice change -- thank you @berkaysynnada

}

/// Splits a string which consists of multiple queries.
pub(crate) fn split_from_semicolon(sql: String) -> Vec<String> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can probably avoid a copy here if you passed in a reference

Suggested change
pub(crate) fn split_from_semicolon(sql: String) -> Vec<String> {
pub(crate) fn split_from_semicolon(sql: &str) -> Vec<&str> {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot find a copy here. If pass the reference, I also need to copy it for exec_and_print().

@alamb alamb changed the title CLI Sequential "Create External Table" Command Support datafusion-cli support for multiple commands in a single "line Mar 28, 2024
@alamb alamb changed the title datafusion-cli support for multiple commands in a single "line datafusion-cli support for multiple commands in a single line Mar 28, 2024

/// Splits a string which consists of multiple queries.
pub(crate) fn split_from_semicolon(sql: String) -> Vec<String> {
sql.split(';')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to add a test? Just wondering if it will work for query

select ';'

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will send a commit excluding semicolons if they are in "" or '', and some tests.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an improvement to me. Thanks @berkaysynnada

Ok(())
}

#[test]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thank you for the tests 👍

let mut in_single_quote = false;
let mut in_double_quote = false;

for c in sql.chars() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely pretty brittle (likely won't handle all crazy corner cases, like for example when there are escaped quotes \\', but I think it is better than what we have now.

Thanks @berkaysynnada

@mustafasrepo mustafasrepo merged commit d795763 into apache:main Mar 29, 2024
Lordworms pushed a commit to Lordworms/arrow-datafusion that referenced this pull request Apr 1, 2024
…#9831)

* Multiple Create External Table's are supported from CLI

* Handle in-quote semicolons

* add test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants