Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for errors in prometheus connector when backquotes are used for fieldnames in aggregations. #1041

Merged
merged 1 commit into from
Nov 8, 2022

Conversation

vamsi-amazon
Copy link
Member

@vamsi-amazon vamsi-amazon commented Nov 4, 2022

Signed-off-by: vamsi-amazon reddyvam@amazon.com

Description

Backquotes in stats groupby list is causing errors in prometheus connector.
Eg command:

source=prometheus.promhttp_metric | stats avg(@value) by span(@timestamp, 15s), `instance`

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@vamsi-amazon vamsi-amazon marked this pull request as ready for review November 4, 2022 22:18
@vamsi-amazon vamsi-amazon requested a review from a team as a code owner November 4, 2022 22:18
@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2022

Codecov Report

Merging #1041 (77588fd) into 2.4 (4a9cef3) will decrease coverage by 2.57%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                2.4    #1041      +/-   ##
============================================
- Coverage     98.26%   95.68%   -2.58%     
- Complexity     3326     3332       +6     
============================================
  Files           324      334      +10     
  Lines          8400     9071     +671     
  Branches        553      672     +119     
============================================
+ Hits           8254     8680     +426     
- Misses          142      334     +192     
- Partials          4       57      +53     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.26% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...us/storage/model/PrometheusResponseFieldNames.java 100.00% <ø> (ø)
...ch/sql/prometheus/response/PrometheusResponse.java 100.00% <100.00%> (ø)
...rage/implementor/PrometheusDefaultImplementor.java 100.00% <100.00%> (ø)
.../storage/querybuilder/AggregationQueryBuilder.java 100.00% <100.00%> (ø)
workbench/public/components/SQLPage/SQLPage.tsx 100.00% <0.00%> (ø)
workbench/public/components/PPLPage/PPLPage.tsx 56.52% <0.00%> (ø)
...ublic/components/QueryResults/QueryResultsBody.tsx 68.32% <0.00%> (ø)
...ch/public/components/QueryResults/QueryResults.tsx 61.60% <0.00%> (ø)
workbench/public/application.tsx 0.00% <0.00%> (ø)
workbench/public/components/app.tsx 0.00% <0.00%> (ø)
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Yury-Fridlyand
Copy link
Collaborator

Backquotes could be used to wrap any identifier. Could you, please, test a more complex scenario?

source=`prometheus`.`promhttp_metric` | stats avg(@value) as `agg` by span(@timestamp, 15s), `instance`, `handler`, `job`

@vamsi-amazon vamsi-amazon force-pushed the backquote-fix branch 2 times, most recently from 63df9f9 to 2c6d0d2 Compare November 4, 2022 22:43
@vamsi-amazon
Copy link
Member Author

Backquotes could be used to wrap any identifier. Could you, please, test a more complex scenario?

source=`prometheus`.`promhttp_metric` | stats avg(@value) as `agg` by span(@timestamp, 15s), `instance`, `handler`, `job`

Sure

ps48
ps48 previously approved these changes Nov 8, 2022
Copy link
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/opensearch-project/sql/blob/2.x/ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java#L365

Is it covered by Parser?

Yes, Kind of covered.

NamedExpression

{
 name :  \`instance\`
 refExpression : {
  attr : "instance"
  }
 }

I can rely on ReferenceExpression instead of stripping the backquotes.

@dai-chen dai-chen added the bug Something isn't working label Nov 8, 2022
YANG-DB
YANG-DB previously approved these changes Nov 8, 2022
anirudha
anirudha previously approved these changes Nov 8, 2022
Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
@vamsi-amazon vamsi-amazon merged commit d3bb902 into opensearch-project:2.4 Nov 8, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 8, 2022
Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit d3bb902)
vamsi-amazon added a commit that referenced this pull request Nov 8, 2022
Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit d3bb902)

Co-authored-by: vamsi-amazon <reddyvam@amazon.com>
dai-chen added a commit that referenced this pull request Nov 9, 2022
* Fix `FLOAT` -> `DOUBLE` cast. (#1025)

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

* Fix error messaging from prometheus. (#1029) (#1037)

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit 4a9cef3)

Co-authored-by: vamsi-amazon <reddyvam@amazon.com>

* Add `query` function as alternate syntax to `query_string` function (#1010)

This maintains backwards compatibility with the v1 engine.

* Update DATE and TIME functions to parse string input as datetime (#991)

Add option to accept datetime like string in both TIME and DATE (eg. accept "1999-01-02 12:12:12" for both TIME and DATE.

Strict check on date for testing for valid dates (eg. Don't accept Feb 30th as a valid date) and throws a SemanticCheckException.

Co-authored-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>

* back quote fix (#1041) (#1050)

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit d3bb902)

Co-authored-by: vamsi-amazon <reddyvam@amazon.com>

* Catalog to Datasource changes (#1027) (#1049)

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit 3e30379)

* Bump jackson to 2.14.0 (#1058)

Signed-off-by: Joshua Li <joshuali925@gmail.com>
(cherry picked from commit 5a1adb2)

* Add valueOf() to Expression (#1055)

Signed-off-by: Joshua Li <joshuali925@gmail.com>

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
Co-authored-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: vamsi-amazon <reddyvam@amazon.com>
Co-authored-by: GabeFernandez310 <64448015+GabeFernandez310@users.noreply.github.com>
Co-authored-by: MitchellGale-BitQuill <104795536+MitchellGale-BitQuill@users.noreply.github.com>
Co-authored-by: Joshua Li <joshuali925@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants