Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(query): integrate "like" to filter execution #15233

Merged
merged 8 commits into from
Apr 17, 2024

Conversation

Dousir9
Copy link
Member

@Dousir9 Dousir9 commented Apr 15, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR integrates like into the filter execution framework, so that like can take advantage of ByPass optimization.
Performance test: a SQL composed of partial statements of TPC-H Q13: select count(*) from orders where o_comment like '%pending%deposits% ' and o_totalprice <= 300000, main 2.6s -> pr 2.2s,

FlameGraph

Before
截屏2024-04-16 10 43 52

After
截屏2024-04-16 10 44 53

  • Fixes #[Link the issue here]

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Apr 15, 2024
Copy link

what-the-diff bot commented Apr 15, 2024

PR Summary

  • Added memchr Dependency
    A new dependency was added to our project to enhance its functionalities.

  • Creation of 'like.rs' and Function Export Changes
    A new file called like.rs was introduced under src/query/expression/src/filter directory, encasing several new functions related to the like functionality.

  • Select Modifications for 'like' Operation
    Changes were made to select.rs and select_expr.rs files, enabling the system to handle and process operations implying the like pattern.

  • Addition of 'select_column_like' Method
    A new method named select_column_like was added to the select_column_scalar.rs file. This allows us to select indices based on a like pattern.

  • Introduction of select_like Method to Selector Struct
    The Selector structure can now process the 'Like' case through the select_like method.

  • Addition of Various Functions for Pattern Compatibility
    Functions named is_like_pattern_escape, check_pattern_type, and PatternType were added to the comparison module. These function checks and identify whether the given pattern is compatible with the like operation or not.

  • Enabled 'let_chains' Feature
    The let_chains feature was added to the main crate enhancing its capabilities.

  • index_unchecked_bytes Method Added to StringColumn struct
    This method allows users to access data from StringColumn structure more effectively.

  • Removed Redundant Functions
    The previously mentioned functions is_like_pattern_escape, check_pattern_type, and PatternType that were initially placed in the scalars module in the mod.rs file, have been removed to maintain efficiency and avoid redundancy. These functions are now incorporated in the comparison module.

@Dousir9 Dousir9 marked this pull request as ready for review April 16, 2024 09:00
src/query/expression/src/filter/like.rs Outdated Show resolved Hide resolved
@Dousir9 Dousir9 enabled auto-merge April 17, 2024 06:10
@Dousir9 Dousir9 added this pull request to the merge queue Apr 17, 2024
github-merge-queue bot pushed a commit that referenced this pull request Apr 17, 2024
* feat(query): integrate "like" to filter execution

* chore: refine code

* chore: update

* chore: format Cargo.toml

* chore: refine code and add test

* chore: remove useless code
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Apr 17, 2024
@BohuTANG BohuTANG merged commit f0cdb98 into datafuselabs:main Apr 17, 2024
72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants