Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport] [2.x] [Concurrent Segment Search]: Implement concurrent aggregations support without profile option #7948

Merged
merged 2 commits into from
Jun 8, 2023

Conversation

sohami
Copy link
Collaborator

@sohami sohami commented Jun 7, 2023

Description

Related Issues

Resolves #7353

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

github-actions bot commented Jun 7, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.pit.DeletePitMultiNodeIT.testDeleteWhileSearch
      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString
      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

@codecov
Copy link

codecov bot commented Jun 8, 2023

Codecov Report

Merging #7948 (f4572f2) into 2.x (740956a) will decrease coverage by 0.03%.
The diff coverage is 73.99%.

@@             Coverage Diff              @@
##                2.x    #7948      +/-   ##
============================================
- Coverage     70.59%   70.57%   -0.03%     
- Complexity    56485    56521      +36     
============================================
  Files          4673     4681       +8     
  Lines        267919   268039     +120     
  Branches      39646    39655       +9     
============================================
+ Hits         189149   189170      +21     
- Misses        62452    62589     +137     
+ Partials      16318    16280      -38     
Impacted Files Coverage Δ
...pensearch/action/search/TransportSearchAction.java 70.99% <0.00%> (-0.22%) ⬇️
...search/aggregations/SearchContextAggregations.java 100.00% <ø> (ø)
...nsearch/search/internal/FilteredSearchContext.java 9.17% <0.00%> (-0.09%) ⬇️
.../org/opensearch/search/internal/SearchContext.java 37.20% <ø> (ø)
...main/java/org/opensearch/search/SearchService.java 71.71% <25.00%> (ø)
...arch/aggregations/DefaultAggregationProcessor.java 48.78% <48.78%> (ø)
...va/org/opensearch/search/DefaultSearchContext.java 78.68% <50.00%> (-0.24%) ⬇️
...h/aggregations/ConcurrentAggregationProcessor.java 70.00% <70.00%> (ø)
.../GlobalAggCollectorManagerWithSingleCollector.java 71.42% <71.42%> (ø)
...arch/aggregations/AggregationCollectorManager.java 73.33% <73.33%> (ø)
... and 15 more

... and 454 files with indirect coverage changes

sohami and others added 2 commits June 7, 2023 21:00
…t without profile option (opensearch-project#7514)

* Refactoring of AggregationReduceContext to use in SearchContext. This will be used for performing shard level reduce of aggregations during concurrent segment search usecase

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Support for non global aggregations with concurrent segment search. This PR does not include the support for
profile option with aggregations to work with concurrent model

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Implement AggregationCollectorManager's reduce

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Use CollectorManager for both concurrent and non concurrent use case
Add CollectorManager for Global Aggregations to support concurrent use case

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address review comments

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address serializing pipelineTree in QueryResult in 2.x for bwc and fix the new tests as well

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

---------

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
@github-actions
Copy link
Contributor

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

@reta reta changed the title [Concurrent Segment Search]: Implement concurrent aggregations support without profile option [Backport] [2.x] [Concurrent Segment Search]: Implement concurrent aggregations support without profile option Jun 8, 2023
@reta reta added the backport PRs or issues specific to backporting features or enhancments label Jun 8, 2023
@reta reta merged commit ed27e72 into opensearch-project:2.x Jun 8, 2023
12 checks passed
gaiksaya pushed a commit to gaiksaya/OpenSearch that referenced this pull request Jun 26, 2023
…gregations support without profile option (opensearch-project#7948)

* [Concurrent Segment Search]: Implement concurrent aggregations support without profile option (opensearch-project#7514)

* Refactoring of AggregationReduceContext to use in SearchContext. This will be used for performing shard level reduce of aggregations during concurrent segment search usecase

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Support for non global aggregations with concurrent segment search. This PR does not include the support for
profile option with aggregations to work with concurrent model

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Implement AggregationCollectorManager's reduce

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Use CollectorManager for both concurrent and non concurrent use case
Add CollectorManager for Global Aggregations to support concurrent use case

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address review comments

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

* Address serializing pipelineTree in QueryResult in 2.x for bwc and fix the new tests as well

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

---------

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>

* Address review comments

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>

---------

Signed-off-by: Sorabh Hamirwasia <sohami.apache@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport PRs or issues specific to backporting features or enhancments
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants