Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reorder semi anti joins #11573

Closed
wants to merge 83 commits into from
Closed

Reorder semi anti joins #11573

wants to merge 83 commits into from

Commits on Jan 25, 2024

  1. I think we can reorder semi joins

    @Tmonster
    Tmonster committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    18b7e51 View commit details
    Browse the repository at this point in the history

  2. I have access to the left and right sets and the join type. Now I sho… 

    …uld be able to reorder the semi join
    @Tmonster
    Tmonster committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    77d503e View commit details
    Browse the repository at this point in the history

  3. can now estimate a semi join to be 20% of the left table

    @Tmonster
    Tmonster committed Jan 25, 2024
    Configuration menu
    Copy the full SHA
    34ad120 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2024

  1. have most everything working. some CEs not quite. Q05 tpch needs to b… 

    …e looked at
    @Tmonster
    Tmonster committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    d8f8f9f View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into reorder-semi-and-an… 

    …ti-joins
    @Tmonster
    Tmonster committed Jan 26, 2024
    Configuration menu
    Copy the full SHA
    8e6b730 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2024

  1. some debugging statements

    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    4ef7415 View commit details
    Browse the repository at this point in the history

  2. make format-fix

    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    d0e4892 View commit details
    Browse the repository at this point in the history

  3. fix code where numerator relations were not properly being merged. Al… 

    …so add better heuristics for determining a distinct count
    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    9eecc85 View commit details
    Browse the repository at this point in the history

  4. add cross product join type

    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    2732612 View commit details
    Browse the repository at this point in the history

  5. fix join is reorderabe function. generate enums

    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    bd1d164 View commit details
    Browse the repository at this point in the history

  6. pausing point

    @Tmonster
    Tmonster committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    8be872d View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2024

  1. fix last join issue and better min/max stats help on distinct count

    @Tmonster
    Tmonster committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    b9885f6 View commit details
    Browse the repository at this point in the history

  2. remove min max stats changesg

    @Tmonster
    Tmonster committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    e7eaf10 View commit details
    Browse the repository at this point in the history

  3. add back in removed test

    @Tmonster
    Tmonster committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    2183142 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. pausing to work on adding benchmarks for ingestion

    @Tmonster
    Tmonster committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    7f49a57 View commit details
    Browse the repository at this point in the history

  2. trying to fix this column lifetime analyzer bug

    @Tmonster
    Tmonster committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    206109e View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. you can also change the bindings filters return. but that's not a goo… 

    …d idea
    @Tmonster
    Tmonster committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    830cdec View commit details
    Browse the repository at this point in the history

  2. remove changing what bindings filters return

    @Tmonster
    Tmonster committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    dfab1b6 View commit details
    Browse the repository at this point in the history

  3. check tests one more time. still need to figure out logical as of joi… 

    …n reordering
    @Tmonster
    Tmonster committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    72cd1be View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2024

  1. Merge branch 'reorder-semi-and-anti-joins' of github.com:Tmonster/duc… 

    …kdb into reorder-semi-and-anti-joins
    @Tmonster
    Tmonster committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    990b296 View commit details
    Browse the repository at this point in the history

  2. make format-fix

    @Tmonster
    Tmonster committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    333f252 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. might be a better solution, lets see if debug passes

    @Tmonster
    Tmonster committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    7d7690c View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. Merge remote-tracking branch 'upstream/main' into reorder_semi_anti_j… 

    …oins_easier_fix
    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    4414834 View commit details
    Browse the repository at this point in the history

  2. remove debugging statements

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    b22940f View commit details
    Browse the repository at this point in the history

  3. fix logic for cross product relations

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    8f8edbd View commit details
    Browse the repository at this point in the history

  4. skip test that I will remove eventually

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    d651dcf View commit details
    Browse the repository at this point in the history

  5. these ideas dont work tbh

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    1fdb318 View commit details
    Browse the repository at this point in the history

  6. so close, but cant figure it out yet

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    8fc0048 View commit details
    Browse the repository at this point in the history

  7. still need to track down this bug

    @Tmonster
    Tmonster committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    9827732 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2024

  1. follow this path. Somewhere in the test visit_operator_expressions_in… 

    …_column_lifetime_analyzer you will find why operator expressions need to be visited first. or why rilters cannot be removed
    @Tmonster
    Tmonster committed Feb 14, 2024
    Configuration menu
    Copy the full SHA
    3ee5e65 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. I think we can reorder semi joins

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    49548c1 View commit details
    Browse the repository at this point in the history

  2. I have access to the left and right sets and the join type. Now I sho… 

    …uld be able to reorder the semi join
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    4ad73e1 View commit details
    Browse the repository at this point in the history

  3. can now estimate a semi join to be 20% of the left table

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    f665ea7 View commit details
    Browse the repository at this point in the history

  4. have most everything working. some CEs not quite. Q05 tpch needs to b… 

    …e looked at
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    6c39fb1 View commit details
    Browse the repository at this point in the history

  5. some debugging statements

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    c45bc52 View commit details
    Browse the repository at this point in the history

  6. make format-fix

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    c9bbc47 View commit details
    Browse the repository at this point in the history

  7. fix code where numerator relations were not properly being merged. Al… 

    …so add better heuristics for determining a distinct count
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    7f1ab5e View commit details
    Browse the repository at this point in the history

  8. add cross product join type

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    57a21f2 View commit details
    Browse the repository at this point in the history

  9. fix join is reorderabe function. generate enums

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    30983ec View commit details
    Browse the repository at this point in the history

  10. pausing point

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    818bceb View commit details
    Browse the repository at this point in the history

  11. fix last join issue and better min/max stats help on distinct count

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    8b250a7 View commit details
    Browse the repository at this point in the history

  12. remove min max stats changesg

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    168e1d0 View commit details
    Browse the repository at this point in the history

  13. add back in removed test

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    bf24aed View commit details
    Browse the repository at this point in the history

  14. pausing to work on adding benchmarks for ingestion

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    7802160 View commit details
    Browse the repository at this point in the history

  15. trying to fix this column lifetime analyzer bug

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    be6f854 View commit details
    Browse the repository at this point in the history

  16. you can also change the bindings filters return. but that's not a goo… 

    …d idea
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    54d13eb View commit details
    Browse the repository at this point in the history

  17. remove changing what bindings filters return

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    23a1854 View commit details
    Browse the repository at this point in the history

  18. check tests one more time. still need to figure out logical as of joi… 

    …n reordering
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    d57c740 View commit details
    Browse the repository at this point in the history

  19. make format-fix

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    3108292 View commit details
    Browse the repository at this point in the history

  20. might be a better solution, lets see if debug passes

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    5afbd5b View commit details
    Browse the repository at this point in the history

  21. remove debugging statements

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    75f6d9d View commit details
    Browse the repository at this point in the history

  22. fix logic for cross product relations

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    1d0b442 View commit details
    Browse the repository at this point in the history

  23. skip test that I will remove eventually

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    8012a31 View commit details
    Browse the repository at this point in the history

  24. these ideas dont work tbh

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    c8d88e0 View commit details
    Browse the repository at this point in the history

  25. so close, but cant figure it out yet

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    21817fe View commit details
    Browse the repository at this point in the history

  26. still need to track down this bug

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    fbd8162 View commit details
    Browse the repository at this point in the history

  27. follow this path. Somewhere in the test visit_operator_expressions_in… 

    …_column_lifetime_analyzer you will find why operator expressions need to be visited first. or why rilters cannot be removed
    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    996fd64 View commit details
    Browse the repository at this point in the history

  28. skip failing test

    @Tmonster
    Tmonster committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    14dcd16 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. Merge branch 'feature' into reorder_semi_anti_joins_easier_fix

    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    d9441ad View commit details
    Browse the repository at this point in the history

  2. no longer fails that test

    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    922fb49 View commit details
    Browse the repository at this point in the history

  3. disable other test, see what else fails

    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    ee258b3 View commit details
    Browse the repository at this point in the history

  4. make format-fix

    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    9910429 View commit details
    Browse the repository at this point in the history

  5. disable other failing test, see what else fails

    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    a08db68 View commit details
    Browse the repository at this point in the history

  6. basically everything works now. Just need to fix the cardinality esti… 

    …mator which is segfaulting now
    @Tmonster
    Tmonster committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    46d0114 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. fix join order optimizer seg fault failure

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    a26d06b View commit details
    Browse the repository at this point in the history

  2. remove/clean up tests

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    a3adc84 View commit details
    Browse the repository at this point in the history

  3. clean up some unused code

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    5076c09 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'reorder_semi_anti_joins_easier_fix' of github.com:Tmons… 

    …ter/duckdb into reorder_semi_anti_joins_easier_fix
    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    9ab4ca4 View commit details
    Browse the repository at this point in the history

  5. remove randomly failing test

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    5a00256 View commit details
    Browse the repository at this point in the history

  6. more code clean up

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    faa0755 View commit details
    Browse the repository at this point in the history

  7. make format-fix

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    71da791 View commit details
    Browse the repository at this point in the history

  8. comment out ensuring statististcs because arrow has not statistics

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    e4352af View commit details
    Browse the repository at this point in the history

  9. make format-fix (again)

    @Tmonster
    Tmonster committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    1cb53ad View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. rewrite mark join needs to be in more places

    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    4d7b82e View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/feature' into reorder_semi_ant… 

    …i_joins_easier_fix
    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    e4d78f4 View commit details
    Browse the repository at this point in the history

  3. distinct count should be max of 1 and the reported distinct count

    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    9e538dd View commit details
    Browse the repository at this point in the history

  4. removed unused variable

    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    d9df4df View commit details
    Browse the repository at this point in the history

  5. clean up test fileg

    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    e01ab55 View commit details
    Browse the repository at this point in the history

  6. remove hashtag

    @Tmonster
    Tmonster committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    790bd63 View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. clang tidy fixes #1

    @Tmonster
    Tmonster committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    b6bb52a View commit details
    Browse the repository at this point in the history

  2. make format-fixes 2

    @Tmonster
    Tmonster committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    c5a2a42 View commit details
    Browse the repository at this point in the history

  3. pr comments 1

    @Tmonster
    Tmonster committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    c9f56d1 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. add more comments for readability

    @Tmonster
    Tmonster committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    7dc4bcb View commit details
    Browse the repository at this point in the history