Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport 2.x] Fix SegmentReplicationUsingRemoteStoreIT#testDropPrimaryDuringReplication. (#9471) #9648

Merged
merged 1 commit into from
Aug 31, 2023
Merged

[Backport 2.x] Fix SegmentReplicationUsingRemoteStoreIT#testDropPrimaryDuringReplication. (#9471) #9648

merged 1 commit into from
Aug 31, 2023

Conversation

mch2
Copy link
Member

@mch2 mch2 commented Aug 30, 2023

Manual backport of #9471.

@mch2
Fix SegmentReplicationUsingRemoteStoreIT#testDropPrimaryDuringReplica…
26230e3 
…tion. (opensearch-project#9471)

* Fix SegmentReplicationUsingRemoteStoreIT#testDropPrimaryDuringReplication.

This test is failing because a concurrent flush can wipe out an old commit file
while we are in the remote store refresh listener. The listener will fetch the latest infos from the reader which will reference a segments_n tht has been deleted by an incoming flush.

To fix this, InternalEngine will preserve the latest commit until a new commit is loaded onto the readerManager.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* update InternalEngine to preserve commit file until a new commit is refreshed on.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Update ReadOnlyEngine inside of resetEngineToGlobalCheckpoint to implement getSegmentInfosSnapshot.
This ensures access to this function is not permitted on the ReadOnlyEngine and is delegated to the new IE once opened.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Update javadoc.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* spotless.

Signed-off-by: Marc Handalian <handalm@amazon.com>

---------

Signed-off-by: Marc Handalian <handalm@amazon.com>
(cherry picked from commit 6cd576f)
@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change 26230e3

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@mch2
Copy link
Member Author

mch2 commented Aug 30, 2023

Gradle Check (Jenkins) Run Completed with:

#9580

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Aug 31, 2023

Codecov Report

Merging #9648 (26230e3) into 2.x (945c4b5) will increase coverage by 0.09%.
Report is 2 commits behind head on 2.x.
The diff coverage is 78.40%.

@@             Coverage Diff              @@
##                2.x    #9648      +/-   ##
============================================
+ Coverage     70.74%   70.84%   +0.09%     
- Complexity    57629    57696      +67     
============================================
  Files          4769     4770       +1     
  Lines        272600   272704     +104     
  Branches      40132    40149      +17     
============================================
+ Hits         192853   193189     +336     
+ Misses        63281    62962     -319     
- Partials      16466    16553      +87
Files Changed Coverage Δ
...ensearch/indices/replication/common/CopyState.java 91.66% <ø> (+2.77%) ⬆️
...in/java/org/opensearch/index/shard/IndexShard.java 69.14% <50.00%> (+0.06%) ⬆️
...a/org/opensearch/action/search/SearchResponse.java 88.42% <70.00%> (-1.21%) ⬇️
...org/opensearch/search/GenericSearchExtBuilder.java 73.84% <73.84%> (ø)
...va/org/opensearch/index/engine/InternalEngine.java 73.63% <85.71%> (+0.47%) ⬆️
...ensearch/action/search/SearchResponseSections.java 97.43% <100.00%> (+1.13%) ⬆️
...search/search/internal/InternalSearchResponse.java 100.00% <100.00%> (ø)

... and 462 files with indirect coverage changes

@tlfeng tlfeng merged commit 62fa576 into opensearch-project:2.x Aug 31, 2023
14 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tlfeng tlfeng tlfeng approved these changes

@reta reta Awaiting requested review from reta reta is a code owner

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

Assignees
No one assigned
Labels
skip-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@mch2 @tlfeng