Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP:[CARBONDATA-2785][ExternalFormat] Optimize table pruning info for pruning by segment #2564

Closed

Commits on Jul 25, 2018

  1. [CARBONDATA-2613] Support csv based carbon table

    1. create csv based carbon table using
    CREATE TABLE fact_table (col1 bigint, col2 string, ..., col100 string)
    STORED BY 'CarbonData'
    TBLPROPERTIES(
      'foramt'='csv',
      'csv.delimiter'=',',
      'csv.header'='col1,col2,col100')
    
    2. Load data to this table using
    ALTER TABLE fact_table ADD SEGMENT LOCATION 'path/to/data1'
    
    This closes apache#2374
    xuchuanyin authored and jackylk committed Jul 25, 2018
    Configuration menu
    Copy the full SHA
    0d2769f View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2018

  1. optimize table pruning info for pruning by segment

    In previous implementation, table pruning is performed once for all
    segments. In this case, the pruning info is updated after default/CG/FG
    datamap pruning.
    
    Now we want the table pruning to be performed segment by segment, so in
    this case, the pruning info will be updated after default/CG/FG datamap
    pruning for each segment.
    
    This means that we need to accumulate the pruning info during pruning.
    xuchuanyin committed Jul 26, 2018
    Configuration menu
    Copy the full SHA
    c8812c5 View commit details
    Browse the repository at this point in the history