Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date as a number in INSERT VALUES may fail or slow down #8129

Closed
den-crane opened this issue Dec 10, 2019 · 3 comments · Fixed by #8231
Closed

Date as a number in INSERT VALUES may fail or slow down #8129

den-crane opened this issue Dec 10, 2019 · 3 comments · Fixed by #8231
Assignees
Labels
comp-formats Input / output formats performance st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken

Comments

@den-crane
Copy link
Contributor

den-crane commented Dec 10, 2019

Probably it's an undefined behavior.
May be better to forbid to insert Date as a number in Values, because TSV does not allow this.

create table vtest(A String, B Int64, C Int64, D Date) Engine=Memory;

19.15.5.18 -- fails if number of values is high

clickhouse-client -q "select 'refcode',4,1,17587 from numbers(1000) format Values">test1k.value

time cat test1k.value |clickhouse-client -q "INSERT INTO vtest Values"
real	0m0.075s

clickhouse-client -q "select 'refcode',4,1,17587 from numbers(1000000) format Values" >test1m.value

cat test1m.value |clickhouse-client -q "INSERT INTO vtest Values"
Code: 27. DB::Exception: Cannot parse input: expected ) before: 87),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\'refcode\',4,1,17587),(\': (at row 5958)

19.18.1.1779 -- 53 seconds!!!

time cat  test1m.value |clickhouse-client -q "INSERT INTO vtest Values"
real	0m53.000s
@den-crane den-crane added the bug Confirmed user-visible misbehaviour in official release label Dec 10, 2019
@filimonov filimonov added comp-formats Input / output formats performance labels Dec 12, 2019
@filimonov
Copy link
Contributor

Maybe related to #6781

/cc @tavplubix

@tavplubix
Copy link
Member

tavplubix commented Dec 12, 2019

It should work fine with enabled input_format_values_deduce_templates_of_expressions setting.

$ time cat test1m.value |clickhouse-client -q "INSERT INTO vtest Values" --input_format_values_deduce_templates_of_expressions=1
real	0m0.321s
user	0m0.247s
sys	0m0.051s

https://clickhouse.yandex/docs/en/operations/settings/settings/#settings-input_format_values_deduce_templates_of_expressions

Also inserting date as a number can be forbidden by disabling input_format_values_interpret_expressions setting.

@tavplubix tavplubix removed the bug Confirmed user-visible misbehaviour in official release label Dec 12, 2019
@alexey-milovidov
Copy link
Member

alexey-milovidov commented Dec 12, 2019

@tavplubix Are we ready to enable it by default? You can make a PR with this change.

@alexey-milovidov alexey-milovidov added the st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken label Dec 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-formats Input / output formats performance st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants