Add search node repurpose commands #6517

kotwanikunal · 2023-03-01T02:33:24Z

Description

Add repurpose command support for search node reutilization

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-03-01T02:58:42Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/11839/
CommitID: bc3e16e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-01T03:30:17Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/11840/
CommitID: fa1dac0

codecov-commenter · 2023-03-01T03:32:11Z

Codecov Report

Merging #6517 (8afcc8a) into main (13d336c) will increase coverage by 0.00%.
The diff coverage is 68.22%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff            @@
##               main    #6517   +/-   ##
=========================================
  Coverage     70.79%   70.80%           
- Complexity    59121    59166   +45     
=========================================
  Files          4804     4804           
  Lines        283141   283210   +69     
  Branches      40815    40841   +26     
=========================================
+ Hits         200462   200531   +69     
+ Misses        66229    66208   -21     
- Partials      16450    16471   +21

Impacted Files	Coverage Δ
.../java/org/opensearch/env/NodeRepurposeCommand.java	`75.00% <67.30%> (-14.89%)`	⬇️
.../main/java/org/opensearch/env/NodeEnvironment.java	`77.93% <100.00%> (+1.46%)`	⬆️

... and 502 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

github-actions · 2023-03-01T17:39:24Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/11863/
CommitID: 334f0ca

andrross · 2023-03-01T23:03:31Z

server/src/main/java/org/opensearch/env/NodeRepurposeCommand.java

-            processNoClusterManagerNoDataNode(terminal, dataPaths, env);
-        } else {
-            processClusterManagerNoDataNode(terminal, dataPaths, env);
+        if (DiscoveryNode.isDataNode(env.settings()) == false) {


What if both isDataNode and isSearchNode are false? This will only execute the first branch of the if statement. Is that correct?

The idea here was that it would operate in a step fashion, because the errors are thrown individually - https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/env/NodeEnvironment.java#L386-L396

The clients can perform the repurposing one at a time.
WDYT?

So lets say I have a node that has been used as both a data and a search node and I want to repurpose it to be a cluster manager node. Does that mean I'd have to run the repurpose command twice?

Yep, since the conditions are currently laid out differently within node startup.

But node startup doesn't have anything to do with this repurpose command though, right? If I run this command on a host that had multiple roles but now no longer has any of them, I would expect it to do the right thing and clear everything out. Is that difficult or not possible for some reason?

It's not difficult, just a bunch of combinatorial logic, which I was trying to avoid. I can add it in, shouldn't take a lot of time.

Does it have to be combinatorial? Like why can't it be:

if not data role: clean up data if exists if not search role: clean up search if exists ...

github-actions · 2023-03-10T01:15:28Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12209/
CommitID: 3c93e79
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-10T01:23:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12210/
CommitID: 03a1fbb
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-10T01:39:52Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12212/
CommitID: e26cf31
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-10T02:27:01Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12215/
CommitID: 7b4ca8d
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-10T04:24:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12220/
CommitID: 4d8a6b4
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-03-10T05:35:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/12223/
CommitID: 4d8a6b4
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>

github-actions · 2023-03-10T06:25:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/12225/
CommitID: 8afcc8a

andrross · 2023-03-11T00:43:52Z

server/src/main/java/org/opensearch/env/NodeRepurposeCommand.java

+        if (repurposeData && repurposeSearch) {
+            terminal.println("Node successfully repurposed to no-cluster-manager, no-data and no-search.");
+        } else if (repurposeData) {
+            terminal.println("Node successfully repurposed to no-cluster-manager and no-data.");
+        } else if (repurposeSearch) {
+            terminal.println("Node successfully repurposed to no-cluster-manager and no-search.");
+        }


Seems like you could avoid this explosion of cases with something like:

roleText.add("no-cluster-manager"); if (repurposeData) roleText.add("no-data"); if (repurposeSearch) roleText.add("no-search"); terminal.println("Node successfully repurposed to: " + String.join(",", roleText));

Same applies in the other cases though might require more refactoring. What do you think?

I thought of doing that, but this makes it more readable and traceable. Given its use case and frequency of use, I think we would rather benefit from this approach.

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> (cherry picked from commit 71a76da) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

(cherry picked from commit 71a76da) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Mingshi Liu <mingshl@amazon.com>

kotwanikunal requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, mch2, nknize, owaiskazi19, adnapibar, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng, VachaShah and xuezhou25 as code owners March 1, 2023 02:33

kotwanikunal added backport 2.x Backport to 2.x branch skip-changelog labels Mar 1, 2023

kotwanikunal force-pushed the cache-reservation branch from bc3e16e to fa1dac0 Compare March 1, 2023 03:01

kotwanikunal force-pushed the cache-reservation branch from fa1dac0 to 334f0ca Compare March 1, 2023 17:06

kotwanikunal removed the skip-changelog label Mar 1, 2023

andrross reviewed Mar 1, 2023

View reviewed changes

kotwanikunal force-pushed the cache-reservation branch 3 times, most recently from 03a1fbb to e26cf31 Compare March 10, 2023 01:13

kotwanikunal force-pushed the cache-reservation branch from e26cf31 to 7b4ca8d Compare March 10, 2023 02:02

kotwanikunal force-pushed the cache-reservation branch from 7b4ca8d to 4d8a6b4 Compare March 10, 2023 03:58

Add search node repurpose commands

8afcc8a

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>

kotwanikunal force-pushed the cache-reservation branch from 4d8a6b4 to 8afcc8a Compare March 10, 2023 05:55

andrross reviewed Mar 11, 2023

View reviewed changes

andrross approved these changes Mar 11, 2023

View reviewed changes

andrross merged commit 71a76da into opensearch-project:main Mar 11, 2023

opensearch-trigger-bot bot mentioned this pull request Mar 11, 2023

[Backport 2.x] Add search node repurpose commands #6628

Merged

mingshl pushed a commit to mingshl/OpenSearch-Mingshl that referenced this pull request Mar 24, 2023

Add search node repurpose commands (opensearch-project#6517)

e489235

Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Mingshi Liu <mingshl@amazon.com>

kotwanikunal deleted the cache-reservation branch May 12, 2023 19:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add search node repurpose commands #6517

Add search node repurpose commands #6517

kotwanikunal commented Mar 1, 2023

github-actions bot commented Mar 1, 2023

github-actions bot commented Mar 1, 2023

codecov-commenter commented Mar 1, 2023 •

edited

github-actions bot commented Mar 1, 2023

andrross Mar 1, 2023

kotwanikunal Mar 9, 2023

andrross Mar 9, 2023

kotwanikunal Mar 9, 2023

andrross Mar 9, 2023

kotwanikunal Mar 9, 2023

andrross Mar 9, 2023

kotwanikunal Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

github-actions bot commented Mar 10, 2023

andrross Mar 11, 2023

kotwanikunal Mar 11, 2023

Add search node repurpose commands #6517

Add search node repurpose commands #6517

Conversation

kotwanikunal commented Mar 1, 2023

Description

Issues Resolved

Check List

github-actions bot commented Mar 1, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 1, 2023

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Mar 1, 2023 • edited

Codecov Report

github-actions bot commented Mar 1, 2023

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Mar 10, 2023

Gradle Check (Jenkins) Run Completed with:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Mar 1, 2023 •

edited