Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: ByConity Projection #347

Merged
merged 5 commits into from Jun 15, 2023
Merged

feat: ByConity Projection #347

merged 5 commits into from Jun 15, 2023

Conversation

gaoyuanning
Copy link

Changelog category :

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add projection module for ByConity.

Documentation entry for user-facing changes

Projection concept

  • Projection rearranges data by different columns
  • Projection defines a pre-aggregated model directly on the source table using an aggregated query
  • Projection query analysis can automatically select the optimal Projection for query optimization without rewriting the query
  • Projection provides consistency guarantees for any data transformation operation at any time

Usage

-- Create a new projection
ALTER TABLE [db].table ADD PROJECTION name ( SELECT <COLUMN LIST EXPR> [GROUP BY] [ORDER BY] )

-- Drop a projection and delete the corresponding data
ALTER TABLE [db].table DROP PROJECTION name 

-- Materialize a projection
ALTER TABLE [db.]table MATERIALIZE PROJECTION name [IN PARTITION partition_name]

gaoyuanning added 5 commits June 12, 2023 20:57
Merge branch 'cnch-ce-projection' into 'cnch-ce-merge'
959ea05 
feat(clickhouse@8787367): CNCH Projection

See merge request dp/ClickHouse!11305
# Conflicts:
#	src/Optimizer/ExpressionDeterminism.cpp
#	src/Optimizer/ExpressionDeterminism.h
#	src/Optimizer/Rule/Rewrite/PushIntoTableScanRules.cpp
#	src/QueryPlan/TableScanStep.cpp
#	src/Storages/MergeTree/MergeTreeCNCHDataDumper.cpp
#	src/Storages/MergeTree/MergeTreeDataPartCNCH.cpp
#	src/WorkerTasks/MergeTreeDataMerger.cpp
#	src/WorkerTasks/MergeTreeDataMerger.h
#	tests/queries/0_stateless/00982_materialized_view_match_rewrite_optimizer.reference
Merge branch 'cnch-ce-projection-loading' into 'cnch-ce-merge'
67c633d 
fix(clickhousech@9677809): Optimize the loading process of CNCH Projections

See merge request dp/ClickHouse!11577
# Conflicts:
#	src/Protos/DataModelHelpers.cpp
Merge branch 'projection_attach' into 'cnch-ce-merge'
f51f5c3 
fix(clickhouse@9677809): attach parts with projection

See merge request dp/ClickHouse!11767
# Conflicts:
#	src/Storages/MergeTree/MergeTreeDataPartCNCH.cpp
#	tests/queries/4_cnch_stateless/40022_topn_filtering_opt.sql
Merge branch 'fix_detached_part_name' into 'cnch-ce-merge'
b1fbc03 
fix(clickhousech@12337924): Fix detached part name

See merge request dp/ClickHouse!12861
fix minor issues
014ecc4
@CLAassistant
Copy link

CLAassistant commented Jun 13, 2023 •
edited

CLA assistant check
All committers have signed the CLA.

@gaoyuanning gaoyuanning changed the title By conity GitHub ByConity Projection Jun 13, 2023
@gaoyuanning gaoyuanning changed the title ByConity Projection feat: ByConity Projection Jun 13, 2023
@hustnn hustnn merged commit 6aceb7b into ByConity:master Jun 15, 2023
2 checks passed
@zhaojintaozhao
Copy link
Contributor

@gaoyuanning : Does Projectoin support S3?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hustnn hustnn hustnn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@gaoyuanning @CLAassistant @zhaojintaozhao @hustnn