Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow nest range expr in Range Query #2557

Merged
merged 7 commits into from
Oct 18, 2023
Merged

feat: allow nest range expr in Range Query #2557

merged 7 commits into from
Oct 18, 2023

Conversation

Taylor-lagrange
Copy link
Collaborator

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Nest expr

A range expression like max(a+1) Range '5m' FILL NULL can be nested behind any expression, but the expression must contain at least one aggregate function.

The following SQL is valid and is semantically equivalent to the following subquery

Select
    round(max(a+1) Range '5m' FILL NULL),
    sin((max(a) + 1) Range '5m' FILL NULL),
from
    test
ALIGN '1h' by (b) FILL NULL;
Select round(x), sin(y + 1) from
    (
        select max(a+1) Range '5m' FILL NULL as x, max(a) Range '5m' FILL NULL as y
            from
        test
            ALIGN '1h' by (b) FILL NULL;
    )
;

But nest range query (e.g. SELECT min(max(val) RANGE '20s') RANGE '20s' FROM host ALIGN '10s';) is invalid.

Change rewrite format

Change the rewrite format to range_fn(func, range, fill, byc, [byv], align). Let func be parsed by datafusion to correctly deduce the type of range expression.

FILL Support

Now fill Option is available in range query, We use fill to fill the value that is null after the range query.

Valid Fill Option are:

  1. NULL(default fill option): No operation is performed, null values will be displayed as original
  2. PREV: The previous value after the null value which is sorted by the timeline will be used to fill the null value
  3. LINEAR: The average of the two values before and after the null value which is sorted by the timeline will be used to fill the null value.
  4. x: Any constant of the same type as the null value. All null values will be filled with this value.

If you use the LINEAR strategy to fill an integer variable, the variable type will be implicitly converted to a floating point type.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

GreptimeTeam/sqlparser-rs#6
#1662

@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

Merging #2557 (dc63c90) into develop (3217b56) will decrease coverage by 0.32%.
Report is 2 commits behind head on develop.
The diff coverage is 87.85%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2557      +/-   ##
===========================================
- Coverage    85.14%   84.83%   -0.32%     
===========================================
  Files          738      738              
  Lines       118319   118955     +636     
===========================================
+ Hits        100747   100919     +172     
- Misses       17572    18036     +464     

src/query/src/range_select/plan.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a great job! I have some comments, please take a look.

tests/cases/standalone/common/range/by.result Show resolved Hide resolved
tests/cases/standalone/common/range/by.sql Outdated Show resolved Hide resolved
tests/cases/standalone/common/range/calculate.sql Outdated Show resolved Hide resolved
tests/cases/standalone/common/range/error.sql Outdated Show resolved Hide resolved
tests/cases/standalone/common/range/precisions.sql Outdated Show resolved Hide resolved
src/query/src/range_select/plan.rs Outdated Show resolved Hide resolved
src/query/src/range_select/plan.rs Outdated Show resolved Hide resolved
src/query/src/range_select/plan_rewrite.rs Outdated Show resolved Hide resolved
src/query/src/range_select/plan_rewrite.rs Show resolved Hide resolved
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@waynexia waynexia added this pull request to the merge queue Oct 18, 2023
Merged via the queue into GreptimeTeam:develop with commit 46e106b Oct 18, 2023
11 checks passed
@Taylor-lagrange Taylor-lagrange deleted the feat-range-expr-nest branch October 22, 2023 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants