Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LowCardinality Dictionary as skip index #16707

Open
UnamedRus opened this issue Nov 5, 2020 · 1 comment
Open

LowCardinality Dictionary as skip index #16707

UnamedRus opened this issue Nov 5, 2020 · 1 comment

Comments

@UnamedRus
Copy link
Contributor

Use case

Clickhouse 20.8 
CREATE TABLE test_lc (partition UInt32, key UInt32, value LowCardinality(String)) ENGINE=MergeTree PARTITION BY partition ORDER BY key;
INSERT INTO test_lc SELECT 0 as a, number, 'dasdsadsdsds' FROM numbers(100000000);
INSERT INTO test_lc SELECT 1 as a, number, 'dasdfsagfgdgs' FROM numbers(100000000);
INSERT INTO test_lc SELECT 2 as a, number, 'sadasdasdasdas' FROM numbers(100000000);
INSERT INTO test_lc SELECT 2 as a, number, 'special value' FROM numbers(11);
SELECT * FROM test_lc WHERE value = 'special value';
2020.11.05 15:34:28.897084 [ 5013 ] {9448e6c4-916c-40bd-96d2-dad4c7273372} <Information> executeQuery: Read 300000011 rows, 286.28 MiB in 0.2246097 sec., 1335650290 rows/sec., 1.24 GiB/sec.
2020.11.05 15:34:28.897180 [ 5013 ] {9448e6c4-916c-40bd-96d2-dad4c7273372} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.

11 rows in set. Elapsed: 0.226 sec. Processed 300.00 million rows, 300.19 MB (1.32 billion rows/s., 1.33 GB/s.)

Describe the solution you'd like
Use lowCardinality dictionary to filter out parts which don't have value for which we are searching.
In case of multiple lowCardinality dictionaries for a single part, it can filter out some ranges from certain parts.

Describe alternatives you've considered
Create secondary indices to handle that kind of queries.

@alexey-milovidov
Copy link
Member

PREWHERE works very similarly, although reading only the dictionary part will be better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants