Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support for attribute expressions in selections #129

Merged
merged 27 commits into from
Dec 29, 2021

Conversation

aaron-steinfeld
Copy link
Contributor

@aaron-steinfeld aaron-steinfeld commented Dec 23, 2021

Description

  • Add support for attribute expressions in selections, aggregations.
  • Fix bug where subpath is used in projection.
  • Refactor separate distinct query transformations.

Testing

  • Updated unit tests
  • Ran existing Integration tests

@codecov
Copy link

codecov bot commented Dec 23, 2021

Codecov Report

Merging #129 (9f23278) into main (976986d) will increase coverage by 0.14%.
The diff coverage is 91.71%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #129      +/-   ##
============================================
+ Coverage     81.25%   81.39%   +0.14%     
- Complexity      574      588      +14     
============================================
  Files            53       58       +5     
  Lines          2235     2268      +33     
  Branches        236      234       -2     
============================================
+ Hits           1816     1846      +30     
- Misses          325      327       +2     
- Partials         94       95       +1     
Flag Coverage Δ
integration 81.39% <91.71%> (+0.14%) ⬆️
unit 79.11% <87.57%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...y/service/projection/ProjectionTransformation.java 76.47% <46.66%> (-10.91%) ⬇️
...ore/query/service/AbstractQueryTransformation.java 94.20% <94.20%> (ø)
...xpressionSubpathExistsFilteringTransformation.java 96.36% <96.36%> (ø)
...ypertrace/core/query/service/QueryRequestUtil.java 79.72% <100.00%> (-3.39%) ⬇️
...ertrace/core/query/service/QueryServiceModule.java 100.00% <100.00%> (ø)
...rtrace/core/query/service/QueryTransformation.java 100.00% <100.00%> (ø)
...ore/query/service/QueryTransformationPipeline.java 93.33% <100.00%> (+0.47%) ⬆️
...ttribubteexpression/AttributeExpressionModule.java 100.00% <100.00%> (ø)
...ttributeExpressionNormalizationTransformation.java 100.00% <100.00%> (ø)
.../query/service/pinot/PinotBasedRequestHandler.java 70.86% <100.00%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 976986d...9f23278. Read the comment docs.

@github-actions

This comment has been minimized.

Base automatically changed from feat_api_support_for_attribute_expr to main December 23, 2021 18:14
@github-actions

This comment has been minimized.

@aaron-steinfeld aaron-steinfeld marked this pull request as ready for review December 23, 2021 18:16
@aaron-steinfeld aaron-steinfeld requested a review from a team December 23, 2021 18:16
Co-authored-by: sarthak <sarthak02singhal@gmail.com>
@github-actions

This comment has been minimized.

if (filter.equals(Filter.getDefaultInstance())) {
return Single.just(filter);
private Expression addSubpathOrThrow(Expression projectedExpression, String subpath) {
if (!QueryRequestUtil.isSimpleAttributeExpression(projectedExpression)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give one example or add a test for this when we hit this condition?

A projected attribute is already having subpath?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, will add a test. As a quick example, imagine if we rename a column and so use a simple projection to point one attribute name at another. In this hypothetical, an old name of EVENT.spanTags becomes EVENT.attributes. An arriving attribute expression may still be using the old name, so when we project

       "attributeExpression": {
          "attributeId": "EVENT.spanTags",
          "subpath": "span.kind"
        }

the projection is only based on the attribute itself, so above we first project:

       "attributeExpression": {
          "attributeId": "EVENT.attributes"
        }

then this function is meant to restore the subpath part giving us our final projection:

       "attributeExpression": {
          "attributeId": "EVENT.attributes",
          "subpath": "span.kind"
        }

It currently throws if the projection did not result in another simple attribute expression, since if the result was already a complex expression (either a function or map selection), it would be more difficult/potentially impossible to support (to be honest, I didn't go too far down that path - we don't have that use case at the moment and can cross that bridge when/if we get there).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test added in #132

Sarthak Singhal added 2 commits December 27, 2021 11:33
@@ -226,6 +228,35 @@ public void testQueryWithNotContainsKeyOperator() {
executionContext);
}

@Test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@github-actions

This comment has been minimized.

@@ -96,8 +96,7 @@ public static void setup() throws Exception {
new GenericContainer<>(DockerImageName.parse("hypertrace/pinot-servicemanager:main"))
.withNetwork(network)
.withNetworkAliases("pinot-controller", "pinot-server", "pinot-broker")
.withExposedPorts(8099)
.withExposedPorts(9000)
.withExposedPorts(8099, 9000)
Copy link
Contributor Author

@aaron-steinfeld aaron-steinfeld Dec 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a pre-existing bug with test config, the second one overwrites the first, it takes varargs instead. It was previously working due to a bug in test containers that was fixed when we upgraded to fix vulnerable deps (unexposed ports on the container were still exposed).

@github-actions

This comment has been minimized.

@@ -24,6 +24,7 @@
QueryTransformationContext transformationContext =
new DefaultQueryTransformationContext(tenantId);
return Observable.fromIterable(transformations)
.sorted()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of this PR, we put in the concept of ordered transformations, so this ensures their order (since mutli injection does not).

@github-actions

This comment has been minimized.

@kotharironak
Copy link
Contributor

@aaron-steinfeld let me know if we are upgrading to log4j version? I will re-approve.

@aaron-steinfeld
Copy link
Contributor Author

Will upgrade it in a following PR

@aaron-steinfeld aaron-steinfeld merged commit 641e419 into main Dec 29, 2021
@aaron-steinfeld aaron-steinfeld deleted the attribute-expression-selections-refactor branch December 29, 2021 13:28
@github-actions
Copy link

Unit Test Results

  32 files  +2    32 suites  +2   8s ⏱️ -2s
187 tests +2  187 ✔️ +2  0 💤 ±0  0 ❌ ±0 

Results for commit 641e419. ± Comparison against base commit 976986d.

@aaron-steinfeld
Copy link
Contributor Author

Will upgrade it in a following PR

#132

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants