Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-4867. Ozone admin datanode list should report dead and stale nodes #1966

Merged
merged 3 commits into from
Mar 1, 2021
Merged

HDDS-4867. Ozone admin datanode list should report dead and stale nodes #1966

merged 3 commits into from
Mar 1, 2021

Conversation

sodonnel
Copy link
Contributor

What changes were proposed in this pull request?

This change adds the health state to the "ozone admin datanode list" command and ensures that all nodes registered in SCM are returned in the output. Previously, the health state was not displayed and only healthy nodes were shown. Any stale or dead nodes were omitted.

The new output looks like:

$ ozone admin datanode list
Datanode: 839e594c-2dd8-4d19-a932-acb2c7acebb9 (/default-rack/172.22.0.4/ozone_datanode_4.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY   ******** Newly added line
Related pipelines:
93c1ecce-e0ed-49ab-be6d-e0c0ec07e3e9/ONE/RATIS/OPEN/Leader
fde8dd62-c677-4a05-89f0-a7d0eb450736/THREE/RATIS/OPEN/Leader

The output is also sorted so healthy DNs are listed first, then stale and then dead at the bottom.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4867

How was this patch tested?

Unit test and confirmed with Docker Compose:

$ ozone admin datanode list
Datanode: 839e594c-2dd8-4d19-a932-acb2c7acebb9 (/default-rack/172.22.0.4/ozone_datanode_4.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
93c1ecce-e0ed-49ab-be6d-e0c0ec07e3e9/ONE/RATIS/OPEN/Leader
fde8dd62-c677-4a05-89f0-a7d0eb450736/THREE/RATIS/OPEN/Leader

Datanode: 8afd6d3e-09e6-40f2-b15e-6809304a9792 (/default-rack/172.22.0.5/ozone_datanode_1.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
4de5a0ef-c13b-4254-b4a6-d4eae3a5bb35/ONE/RATIS/OPEN/Leader
fde8dd62-c677-4a05-89f0-a7d0eb450736/THREE/RATIS/OPEN/Follower

Datanode: 4dc7f867-b1b7-423f-8eb2-3bb46f221bbb (/default-rack/172.22.0.6/ozone_datanode_3.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
fde8dd62-c677-4a05-89f0-a7d0eb450736/THREE/RATIS/OPEN/Follower
a99d5f34-e36a-47b6-8816-744bc719716d/ONE/RATIS/OPEN/Leader

Datanode: a086be73-8450-4506-9446-2d54a96951ff (/default-rack/172.22.0.7/ozone_datanode_2.ozone_default/0 pipelines)
Operational State: IN_SERVICE
Health State: DEAD
Related pipelines:
No related pipelines or the node is not in Healthy state.

adoroszlai
adoroszlai approved these changes Mar 1, 2021
View reviewed changes
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sodonnel for the patch. Tested in ozone-topology environment, works fine.

@sodonnel sodonnel merged commit 44e969d into apache:master Mar 1, 2021
errose28 added a commit to errose28/ozone that referenced this pull request Mar 2, 2021
@errose28
Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonroll…
ce9148f 
…ing-upgrade

* upstream/master: (29 commits)
  HDDS-4741. Modularize upgrade test (apache#1928)
  HDDS-4864. Add acceptance tests to certify Ozone with boto3 python client. (apache#1976)
  HDDS-4791. StateContext.getReports may return list with size larger t… (apache#1892)
  HDDS-4867. Ozone admin datanode list should report dead and stale nodes (apache#1966)
  HDDS-4858. Useless Maven cache cleanup (apache#1956)
  HDDS-4769. Simplify insert operation of ContainerAttribute (apache#1865)
  HDDS-4847. Fix typo in name of IdentityService (apache#1941)
  HDDS-4869. Bump jackson version number (apache#1963)
  HDDS-4871. Fix intellij runConfigurations for datanode (apache#1968)
  HDDS-4870. Bump jetty version (apache#1964)
  HDDS-4722. Creating RDBStore fails due to RDBMetrics instance race (apache#1820)
  HDDS-4138. Improve crc efficiency by using Java.util.zip.CRC when available (apache#1950)
  HDDS-4816. Add UsageInfoSubcommand to get Datanode usage information. (apache#1919)
  HDDS-4754. Make scm heartbeat rpc retry interval configurable (apache#1942)
  HDDS-4832. Show Datanode OperationalState in Recon (apache#1937)
  HDDS-4653. Support TDE for MPU Keys on Encrypted Buckets (apache#1766)
  HDDS-4853. libexec/entrypoint.sh might copy from wrong path (apache#1951)
  HDDS-4857. Format ReplicationType.java which indentation are confusion (apache#1952)
  HDDS-4850. Intermittent failure in ozonesecure due to unable to allocate block (apache#1948)
  HDDS-4808. Add Genesis benchmark for various CRC implementations (apache#1910)
  ...

Conflicts:
	hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/client/ScmClient.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java
	hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java
	hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
	hadoop-hdds/interface-client/src/main/proto/hdds.proto
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
	hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerOperationClient.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
errose28 added a commit to errose28/ozone that referenced this pull request Mar 11, 2021
@errose28
Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonroll…
7eefee7 
…ing-upgrade-merge-candidate

* upstream/master: (29 commits)
  HDDS-4741. Modularize upgrade test (apache#1928)
  HDDS-4864. Add acceptance tests to certify Ozone with boto3 python client. (apache#1976)
  HDDS-4791. StateContext.getReports may return list with size larger t… (apache#1892)
  HDDS-4867. Ozone admin datanode list should report dead and stale nodes (apache#1966)
  HDDS-4858. Useless Maven cache cleanup (apache#1956)
  HDDS-4769. Simplify insert operation of ContainerAttribute (apache#1865)
  HDDS-4847. Fix typo in name of IdentityService (apache#1941)
  HDDS-4869. Bump jackson version number (apache#1963)
  HDDS-4871. Fix intellij runConfigurations for datanode (apache#1968)
  HDDS-4870. Bump jetty version (apache#1964)
  HDDS-4722. Creating RDBStore fails due to RDBMetrics instance race (apache#1820)
  HDDS-4138. Improve crc efficiency by using Java.util.zip.CRC when available (apache#1950)
  HDDS-4816. Add UsageInfoSubcommand to get Datanode usage information. (apache#1919)
  HDDS-4754. Make scm heartbeat rpc retry interval configurable (apache#1942)
  HDDS-4832. Show Datanode OperationalState in Recon (apache#1937)
  HDDS-4653. Support TDE for MPU Keys on Encrypted Buckets (apache#1766)
  HDDS-4853. libexec/entrypoint.sh might copy from wrong path (apache#1951)
  HDDS-4857. Format ReplicationType.java which indentation are confusion (apache#1952)
  HDDS-4850. Intermittent failure in ozonesecure due to unable to allocate block (apache#1948)
  HDDS-4808. Add Genesis benchmark for various CRC implementations (apache#1910)
  ...

Conflicts:
	hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/client/ScmClient.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocol.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java
	hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/StorageContainerLocationProtocolClientSideTranslatorPB.java
	hadoop-hdds/interface-admin/src/main/proto/ScmAdminProtocol.proto
	hadoop-hdds/interface-client/src/main/proto/hdds.proto
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/protocol/StorageContainerLocationProtocolServerSideTranslatorPB.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java
	hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/ContainerOperationClient.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
	hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconNodeManager.java
errose28 added a commit to errose28/ozone that referenced this pull request Mar 16, 2021
@errose28
Merge branch 'HDDS-3698-nonrolling-upgrade' into HDDS-4180
5c64f04 
* HDDS-3698-nonrolling-upgrade: (29 commits)
  HDDS-4741. Modularize upgrade test (apache#1928)
  HDDS-4864. Add acceptance tests to certify Ozone with boto3 python client. (apache#1976)
  HDDS-4791. StateContext.getReports may return list with size larger t… (apache#1892)
  HDDS-4867. Ozone admin datanode list should report dead and stale nodes (apache#1966)
  HDDS-4858. Useless Maven cache cleanup (apache#1956)
  HDDS-4769. Simplify insert operation of ContainerAttribute (apache#1865)
  HDDS-4847. Fix typo in name of IdentityService (apache#1941)
  HDDS-4869. Bump jackson version number (apache#1963)
  HDDS-4871. Fix intellij runConfigurations for datanode (apache#1968)
  HDDS-4870. Bump jetty version (apache#1964)
  HDDS-4722. Creating RDBStore fails due to RDBMetrics instance race (apache#1820)
  HDDS-4138. Improve crc efficiency by using Java.util.zip.CRC when available (apache#1950)
  HDDS-4816. Add UsageInfoSubcommand to get Datanode usage information. (apache#1919)
  HDDS-4754. Make scm heartbeat rpc retry interval configurable (apache#1942)
  HDDS-4832. Show Datanode OperationalState in Recon (apache#1937)
  HDDS-4653. Support TDE for MPU Keys on Encrypted Buckets (apache#1766)
  HDDS-4853. libexec/entrypoint.sh might copy from wrong path (apache#1951)
  HDDS-4857. Format ReplicationType.java which indentation are confusion (apache#1952)
  HDDS-4850. Intermittent failure in ozonesecure due to unable to allocate block (apache#1948)
  HDDS-4808. Add Genesis benchmark for various CRC implementations (apache#1910)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adoroszlai adoroszlai adoroszlai approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sodonnel @adoroszlai