[Remote Store Migration] Skip segrep lag computation for shard copies on docrep nodes #14119
Conversation
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
shourya035
requested review from
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
dreamer-89,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
tlfeng and
VachaShah
as code owners
|
❌ Gradle check result for 819ffae: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
server/src/main/java/org/opensearch/index/seqno/ReplicationTracker.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
|
❌ Gradle check result for 84001dd: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
|
❌ Gradle check result for f0274c7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 3787df2: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 45bfacb: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 23b6173: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❕ Gradle check result for 23b6173: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14119 +/- ##
============================================
+ Coverage 71.42% 71.65% +0.23%
- Complexity 59978 61882 +1904
============================================
Files 4985 5113 +128
Lines 282275 290792 +8517
Branches 40946 41990 +1044
============================================
+ Hits 201603 208374 +6771
- Misses 63999 65217 +1218
- Partials 16673 17201 +528 ☔ View full report in Codecov by Sentry. |
… on docrep nodes (#14119) (#14163) (cherry picked from commit 710d818) Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
During the dual replication phase, we skip creating checkpoint timers for replication group members that are still left behind on docrep nodes and are yet to mover over to remote enabled nodes.
However, the segrep bytes behind calculation logic still considers those shard copies. Since those shards are still on docrep and doesn't understand the Segrep checkpoints, they add up to the bytes_behind computation logic, in turn exhibiting false sense of replication lag during the remote store migration process.
With this PR, if the primary shard:
and
we are skipping those allocationIds while calculating segment replication lag metrics
Added a new IT through which this issue was readily reproduced without the current code change
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
API changes companion pull request created, if applicable.Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.