Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables #37893

Merged
merged 44 commits into from Jul 26, 2022
Merged

Support SQL standard "delete from ... where ..." syntax and lightweight implementation on merge tree tables #37893

merged 44 commits into from Jul 26, 2022

Commits on Jul 15, 2022

  1. Support delete from ... where syntax on mergetree tables

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    a461882 View commit details
    Browse the repository at this point in the history

  2. Fix style check errors

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    192ca25 View commit details
    Browse the repository at this point in the history

  3. Support new mutation type for lightweight

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    8df7b7a View commit details
    Browse the repository at this point in the history

  4. Support lightweight delete execution using string as deleted rows mas… 

    …k,also part of select can handle LWD
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    8696319 View commit details
    Browse the repository at this point in the history

  5. Disable optimizations for count() when lightweight delete exists, add… 

    … hasLightweightDelete() function in IMergeTreeDataPart
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    b4a37e1 View commit details
    Browse the repository at this point in the history

  6. Merge support to apply lightweight delete mask

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    2c74e9b View commit details
    Browse the repository at this point in the history

  7. For some columns mutations, skip to apply deleted mask when read some… 

    … columns. Also add unit test case
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    9d27af7 View commit details
    Browse the repository at this point in the history

  8. Add missing code for deleted_mask_filter_holder

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    11fdea6 View commit details
    Browse the repository at this point in the history

  9. Code changes due to master new fixes, and update reference for mutati… 

    …ons table
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    8ad2bb7 View commit details
    Browse the repository at this point in the history

  10. Fix lightweight delete bugs: add skip files and use source part's col… 

    …umns to avoid metadata updated cases
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    6248486 View commit details
    Browse the repository at this point in the history

  11. Fix code style error

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    dcc7367 View commit details
    Browse the repository at this point in the history

  12. Change deleted rows mask from String to Native UInt8 format

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    7e43385 View commit details
    Browse the repository at this point in the history

  13. Fix compile error in clang tidy build

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    6e6f77e View commit details
    Browse the repository at this point in the history

  14. Code changes based on review opinions. Make ordinary single alter del… 

    …ete lightwight by default
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    780cdfb View commit details
    Browse the repository at this point in the history

  15. Consider lightweight delete as ordinary for compact part in selectPar… 

    …tsToMutate()
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    198674e View commit details
    Browse the repository at this point in the history

  16. code changes for strange name and adding exception for mutation entry

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    9d1dff5 View commit details
    Browse the repository at this point in the history

  17. Add supportLightweightDeleteMutate() in IMergeTreeDataPart to disable… 

    … LWD for part with projections
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    4268ac8 View commit details
    Browse the repository at this point in the history

  18. Remove loadDeletedMask() and get deleted mask when needed

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    d37152a View commit details
    Browse the repository at this point in the history

  19. fix clang tidy compile error

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    8b1114f View commit details
    Browse the repository at this point in the history

  20. Fix clang tidy compile error, remove std::move for const getDeletedRo… 

    …wsPtr()
    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    0793add View commit details
    Browse the repository at this point in the history

  21. Rewrite logic for loading deleted mask related to getDeletedMask()

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    ca42f64 View commit details
    Browse the repository at this point in the history

  22. Killed mutation has empty commands may cause SegV

    @zhangjmruc
    zhangjmruc committed Jul 15, 2022
    Configuration menu
    Copy the full SHA
    197d379 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2022

  1. POC lightweight delete using __row_exists virtual column and prewhere… 

    …-like filtering
    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    9de72d9 View commit details
    Browse the repository at this point in the history

  2. Rewrite DELETE FROM into UPDATE __row_exist=0

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    f956810 View commit details
    Browse the repository at this point in the history

  3. Cleanups

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    f324ca9 View commit details
    Browse the repository at this point in the history

  4. Renamed __row_exists to _row_exists

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    ae0d000 View commit details
    Browse the repository at this point in the history

  5. Speedup 02352_lightweight_delete test

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    ab29e18 View commit details
    Browse the repository at this point in the history

  6. Disable trivial count() optimization if _row_exists column is present

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    614cb9a View commit details
    Browse the repository at this point in the history

  7. Perf test for read after deleting many rows

    @davenger
    davenger committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    f31788e View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2022

  1. Mix lightweight deletes with old-style alter-delete's

    @davenger
    davenger committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    6578d4f View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2022

  1. Leave only _row_exists-based implementation of lightweight delete

    @davenger
    davenger committed Jul 21, 2022
    Configuration menu
    Copy the full SHA
    1ea9f14 View commit details
    Browse the repository at this point in the history

  2. Make lightweight delete work with compact parts (not in optimal way yet)

    @davenger
    davenger committed Jul 21, 2022
    Configuration menu
    Copy the full SHA
    50bd814 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2022

  1. Cleanups

    @davenger
    davenger committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    5941954 View commit details
    Browse the repository at this point in the history

  2. Properly handle empty parts list

    @davenger
    davenger committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    be64b45 View commit details
    Browse the repository at this point in the history

  3. Disable lightweight delete if table has projections

    @davenger
    davenger committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    c8b3c57 View commit details
    Browse the repository at this point in the history

  4. Allow to update "system" virtual columns such as _row_exists

    @davenger
    davenger committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    6692770 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into feature/sql-standard-delete

    @davenger
    davenger committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    460950e View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2022

  1. More detailed comment

    @davenger
    davenger committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    7b44950 View commit details
    Browse the repository at this point in the history

  2. Capitalized const name

    @davenger
    davenger committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    48de02a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into feature/sql-standard-delete

    @davenger
    davenger committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    25deba2 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2022

  1. ReplicatedMergeTree support lightweight delete

    @zhangjmruc
    zhangjmruc committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    c1834d1 View commit details
    Browse the repository at this point in the history

  2. set mutations_sync to 2 to wait for mutations to complete on all repl… 

    …icas
    @zhangjmruc
    zhangjmruc committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    162be5a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into feature/sql-standard-delete

    @davenger
    davenger committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    0666ec2 View commit details
    Browse the repository at this point in the history

  4. Added 'long' tag

    @davenger
    davenger committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    1b1fa54 View commit details
    Browse the repository at this point in the history