Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Projection Optimization Rule #3

Open
wants to merge 105 commits into
base: apache_main
Choose a base branch
from
Open

Projection Optimization Rule #3

wants to merge 105 commits into from

Commits on Feb 15, 2024

  1. Draft PR is moved to new repo

    @berkaysynnada
    berkaysynnada committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    1c02a68 View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. Add new reproducer test

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e8bb633 View commit details
    Browse the repository at this point in the history

  2. Simplifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    fd29eea View commit details
    Browse the repository at this point in the history

  3. Simplifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    4233028 View commit details
    Browse the repository at this point in the history

  4. Simplifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    410f44a View commit details
    Browse the repository at this point in the history

  5. Simplifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    60fea27 View commit details
    Browse the repository at this point in the history

  6. Simpifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    e98326d View commit details
    Browse the repository at this point in the history

  7. Simplifications

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    2af385a View commit details
    Browse the repository at this point in the history

  8. Minor changes

    @mustafasrepo
    mustafasrepo committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    56b3136 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. tpch fails

    @berkaysynnada
    berkaysynnada committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    26542ba View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. Delete docs.yaml

    @metesynnada
    metesynnada committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    10000fb View commit details
    Browse the repository at this point in the history

  2. Merge pull request #5 from synnada-ai/ci-action-fixing 

    Fix deploying DataFusion site error
    @mustafasrepo
    mustafasrepo committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    6c76423 View commit details
    Browse the repository at this point in the history

  3. All tests pass, ready to be cleaned-up

    @berkaysynnada
    berkaysynnada committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    04143ab View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    4eea8da View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. Merge branch 'apache:main' into main

    @mustafasrepo
    mustafasrepo committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    599516b View commit details
    Browse the repository at this point in the history

  2. Prevent projection removals causing uncertain column swaps

    @berkaysynnada
    berkaysynnada committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    b9a2dfb View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2024

  1. Merge branch 'apache:main' into main

    @mustafasrepo
    mustafasrepo committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    b20b65c View commit details
    Browse the repository at this point in the history

  2. Minor changes

    @berkaysynnada
    berkaysynnada committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    f75cf6b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    b931c4e View commit details
    Browse the repository at this point in the history

  4. Minor changes

    @berkaysynnada
    berkaysynnada committed Feb 26, 2024
    Configuration menu
    Copy the full SHA
    8b06852 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e632f7a View commit details
    Browse the repository at this point in the history

  2. Mini Review before unwraps - Part 1

    @metesynnada
    metesynnada committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3bda474 View commit details
    Browse the repository at this point in the history

  3. Continue on review - Part 2

    @metesynnada
    metesynnada committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    4b4054b View commit details
    Browse the repository at this point in the history

  4. Adding comments - Review Part 3

    @metesynnada
    metesynnada committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    5fb5202 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. Clone's and unwrap's are removed

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    1bb2d7e View commit details
    Browse the repository at this point in the history

  2. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    c8f39fe View commit details
    Browse the repository at this point in the history

  3. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    79c8ce5 View commit details
    Browse the repository at this point in the history

  4. add yaml

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    aa296dc View commit details
    Browse the repository at this point in the history

  5. projection update

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    8b46775 View commit details
    Browse the repository at this point in the history

  6. Pushdown through limits

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    4ccab8b View commit details
    Browse the repository at this point in the history

  7. fix clippy

    @berkaysynnada
    berkaysynnada committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    23e4954 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. ProjectionMapping refactor

    @berkaysynnada
    berkaysynnada committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    c9dce82 View commit details
    Browse the repository at this point in the history

  2. Sync all joins

    @berkaysynnada
    berkaysynnada committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    5fe50fc View commit details
    Browse the repository at this point in the history

  3. Adapt test changes

    @berkaysynnada
    berkaysynnada committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    6ed6878 View commit details
    Browse the repository at this point in the history

  4. Remove duplications

    @berkaysynnada
    berkaysynnada committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    f757233 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2024

  1. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    5e2859d View commit details
    Browse the repository at this point in the history

  2. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    12c4307 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    d6b511b View commit details
    Browse the repository at this point in the history

  2. Fix after merge

    @berkaysynnada
    berkaysynnada committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    0aa1426 View commit details
    Browse the repository at this point in the history

  3. Inherit hashjoin projection

    @berkaysynnada
    berkaysynnada committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    fefd91f View commit details
    Browse the repository at this point in the history

  4. Minor changes

    @berkaysynnada
    berkaysynnada committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    464d7bb View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Project csv only with requirements

    @berkaysynnada
    berkaysynnada committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    ca5d843 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. to check diff

    @berkaysynnada
    berkaysynnada committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    b586311 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. too many open files!

    @berkaysynnada
    berkaysynnada committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    0bdd3b7 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2024

  1. tests pass

    @berkaysynnada
    berkaysynnada committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    5b4e107 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    1ad9303 View commit details
    Browse the repository at this point in the history

  3. get upstream changes

    @berkaysynnada
    berkaysynnada committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    e384bb1 View commit details
    Browse the repository at this point in the history

  4. test changes

    @berkaysynnada
    berkaysynnada committed Mar 19, 2024
    Configuration menu
    Copy the full SHA
    023614e View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. Unify hashjoins

    @berkaysynnada
    berkaysynnada committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    e2a41e4 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    b28e0cd View commit details
    Browse the repository at this point in the history

  3. Addressing mete's latest feedback

    @berkaysynnada
    berkaysynnada committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    b209391 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. Fixing the bug during benchmarks

    @berkaysynnada
    berkaysynnada committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    a1f7d30 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    088f72d View commit details
    Browse the repository at this point in the history

  3. Update cte.slt

    @berkaysynnada
    berkaysynnada committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    f5e58e1 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. tests added

    @berkaysynnada
    berkaysynnada committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    7f77d50 View commit details
    Browse the repository at this point in the history

  2. checked till try_remove

    @berkaysynnada
    berkaysynnada committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    9d11497 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    886e50b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    c3673a0 View commit details
    Browse the repository at this point in the history

  3. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    37b4a0c View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. Fix projection renaming

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    9ad9834 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    52e6867 View commit details
    Browse the repository at this point in the history

  3. agg fixes

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    28fcbdc View commit details
    Browse the repository at this point in the history

  4. test updates

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    73147b5 View commit details
    Browse the repository at this point in the history

  5. fixing count mismatch

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    906b73a View commit details
    Browse the repository at this point in the history

  6. Update aggregate_statistics.rs

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    cac1d1b View commit details
    Browse the repository at this point in the history

  7. catch different names

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    2f60c16 View commit details
    Browse the repository at this point in the history

  8. Fix after merge

    @berkaysynnada
    berkaysynnada committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    b650e00 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    ffebca5 View commit details
    Browse the repository at this point in the history

  2. fix after merge

    @berkaysynnada
    berkaysynnada committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    155a53f View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. fixing tpch queries

    @berkaysynnada
    berkaysynnada committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    4d02e9e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    74b8bcd View commit details
    Browse the repository at this point in the history

  3. Update physical_planner.rs

    @berkaysynnada
    berkaysynnada committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    23b306e View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    0bc2421 View commit details
    Browse the repository at this point in the history

  2. Remove unnecessary projections

    @berkaysynnada
    berkaysynnada committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    09bfda2 View commit details
    Browse the repository at this point in the history

  3. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    cef1c36 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Fixing all tests

    @berkaysynnada
    berkaysynnada committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    4f33cf0 View commit details
    Browse the repository at this point in the history

  2. Self review part 1

    @berkaysynnada
    berkaysynnada committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    679b754 View commit details
    Browse the repository at this point in the history

  3. Self review part 2

    @berkaysynnada
    berkaysynnada committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    26cd56c View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Review

    @ozankabak
    ozankabak committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    c63be56 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    41ccce5 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. before test complete

    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    2128475 View commit details
    Browse the repository at this point in the history

  2. joins are fixed

    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    bb06baf View commit details
    Browse the repository at this point in the history

  3. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    ee8dfa6 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    213119d View commit details
    Browse the repository at this point in the history

  5. Ready for review

    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    16d5e5e View commit details
    Browse the repository at this point in the history

  6. Investigation 

    Projections are preferred to be on top of
1) GlobalLimitExec
2) LocalLimitExec
3) CoalesceBatchesExec
Only slt tests are modified. Unit test results have not been updated.
    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    c87dbde View commit details
    Browse the repository at this point in the history

  7. Revert "Investigation" 

    This reverts commit c87dbde.
    @berkaysynnada
    berkaysynnada committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    bd8f2b2 View commit details
    Browse the repository at this point in the history

  8. Review Part 2

    @ozankabak
    ozankabak committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    5d5cf88 View commit details
    Browse the repository at this point in the history

  9. Remove whitespace diffs

    @ozankabak
    ozankabak committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    b826d29 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Do not pushdown projections making calculations

    @berkaysynnada
    berkaysynnada committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    fb8bf5a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    4315494 View commit details
    Browse the repository at this point in the history

  3. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    1611ca1 View commit details
    Browse the repository at this point in the history

  4. Update module comments for optimize_projections.rs

    @ozankabak
    ozankabak committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    1ff7d19 View commit details
    Browse the repository at this point in the history

  5. Review

    @ozankabak
    ozankabak committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    8e2f487 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Unify similar plans

    @berkaysynnada
    berkaysynnada committed May 3, 2024
    Configuration menu
    Copy the full SHA
    7396744 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed May 3, 2024
    Configuration menu
    Copy the full SHA
    5dd4289 View commit details
    Browse the repository at this point in the history

  3. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed May 3, 2024
    Configuration menu
    Copy the full SHA
    8bdd12f View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Refactor ExprMapping

    @ozankabak
    ozankabak committed May 6, 2024
    Configuration menu
    Copy the full SHA
    8fd5570 View commit details
    Browse the repository at this point in the history

  2. Use error macros

    @ozankabak
    ozankabak committed May 6, 2024
    Configuration menu
    Copy the full SHA
    2c7af01 View commit details
    Browse the repository at this point in the history

  3. Simplify try_projection_insertion

    @ozankabak
    ozankabak committed May 6, 2024
    Configuration menu
    Copy the full SHA
    c5d8d97 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. Update optimize_projections.rs

    @berkaysynnada
    berkaysynnada committed May 16, 2024
    Configuration menu
    Copy the full SHA
    4b91806 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'apache_main' into feature/optimize-projections

    @berkaysynnada
    berkaysynnada committed May 16, 2024
    Configuration menu
    Copy the full SHA
    d6c7ad2 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. Fix the bug

    @berkaysynnada
    berkaysynnada committed May 17, 2024
    Configuration menu
    Copy the full SHA
    d209193 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. Update cte.slt

    @berkaysynnada
    berkaysynnada committed May 21, 2024
    Configuration menu
    Copy the full SHA
    e782be9 View commit details
    Browse the repository at this point in the history

  2. Update cte.slt

    @berkaysynnada
    berkaysynnada committed May 21, 2024
    Configuration menu
    Copy the full SHA
    21568c2 View commit details
    Browse the repository at this point in the history