Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Special query plan step for read from MergeTree #22352

Merged
merged 24 commits into from Apr 19, 2021

Conversation

KochetovNicolai
Copy link
Member

@KochetovNicolai KochetovNicolai commented Mar 30, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add setting indexes (boolean, disabled by default) to EXPLAIN PIPELINE query. When enabled, shows used indexes, number of filtered parts and granules for every index applied. Supported for MergeTree* tables.

@robot-clickhouse robot-clickhouse added the pr-not-for-changelog This PR should not be mentioned in the changelog label Mar 30, 2021
@alesapin alesapin changed the title Special query plan ster for read from MergeTree Special query plan step for read from MergeTree Mar 30, 2021
@KochetovNicolai KochetovNicolai marked this pull request as ready for review April 8, 2021 15:03
@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature and removed pr-not-for-changelog This PR should not be mentioned in the changelog labels Apr 8, 2021
@alesapin alesapin self-assigned this Apr 9, 2021
selectPartsToRead(parts, part_values, minmax_idx_condition, minmax_columns_types, partition_pruner, max_block_numbers_to_read);
selectPartsToRead(parts, part_values, minmax_idx_condition, minmax_columns_types, partition_pruner, max_block_numbers_to_read, part_filter_counters);

index_stats->emplace_back(ReadFromMergeTree::IndexStat{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can avoid printing it if we have at least one index condition?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is useful to add.
We can see how indexes drop parts and granules.

Copy link
Member

@alesapin alesapin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, But I have a question. Why we return QueryPlan with a single step? Seems like it would be more clear to return the step from storage?

Name: t_minmax
Parts: 1
Granules: 2
Skip
Copy link
Member

@alesapin alesapin Apr 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see Indexes header, but this Skip looks slightly confusing. Don't have any ideas on how to improve naming here. Maybe we can use something like Skip (minmax) or Skip (set)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will add it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, also it may be confusing cause I grep out the description.
It will be minmax GRANULARITY 2


/// This step is created to read from MergeTree* table.
/// For now, it takes a list of parts and creates source from it.
class ReadFromMergeTree : public ISourceStep
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final ?

InReverseOrder,
};

explicit ReadFromMergeTree(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explicit looks redundant

@KochetovNicolai
Copy link
Member Author

Functional stateless tests (release, wide parts enabled)

2021-04-16 15:33:55 -          Condition: and((bitAnd(z, 3) not in [1, 1]), (y in [1, +inf)))
2021-04-16 15:33:55 +          Condition: and((bitAnd(z, 3) not in [1, 1]), and((y in [1, +inf)), (bitAnd(z, 3) not in [1, 1])))

that's unusual

@KochetovNicolai KochetovNicolai merged commit c596288 into master Apr 19, 2021
@KochetovNicolai KochetovNicolai deleted the add-read-from-mt-step branch April 19, 2021 14:38
@sevirov
Copy link
Contributor

sevirov commented Apr 20, 2021

Internal documentation ticket: DOCSUP-8737.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature Pull request with new product feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants