Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDDS-4866. Datanode with scmID format should work with clusterID dire… #1959

Merged
merged 19 commits into from Mar 4, 2021
Merged

HDDS-4866. Datanode with scmID format should work with clusterID dire… #1959

merged 19 commits into from Mar 4, 2021

Conversation

mukul1987
Copy link
Contributor

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4866

How was this patch tested?

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

@mukul1987
Copy link
Contributor Author

mukul1987 commented Feb 27, 2021 •
edited

The output of the tool when the layout doesn't match

➜ ~/code/apache/ozone/oz_scm_ha/hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT git:(HDDS-4866) bin/ozone debug dnlayout --verify --path=/tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers
2021-02-27 18:22:50,403 [main] INFO fs.SaveSpaceUsageToFile: Cached usage info file /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/scmUsed not found
2021-02-27 18:22:50,406 [main] INFO volume.HddsVolume: Creating Volume: /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds of storage type : DISK and capacity : 499963174912
2021-02-27 18:22:50,409 [main] INFO volume.MutableVolumeSet: Added Volume : /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds to VolumeSet
2021-02-27 18:22:50,416 [main] INFO volume.ThrottledAsyncChecker: Scheduling a check for /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:22:50,431 [main] INFO volume.HddsVolumeChecker: Scheduled health check for volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:22:50,458 [main] INFO ozoneimpl.ContainerReader: Running in upgrade mode:false
2021-02-27 18:22:50,459 [Thread-3] INFO ozoneimpl.ContainerReader: Start to verify containers on volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:22:50,459 [Thread-3] ERROR ozoneimpl.ContainerReader: Caught a Run time exception during reading container files from Volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds {}
java.lang.IllegalArgumentException: Storage Dir:/tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds/510b80d0-fd9a-42f9-ae9a-7828893aedac doesn't exists
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.preProcessStorageLoc(ContainerReader.java:174)
at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.readVolume(ContainerReader.java:143)
at org.apache.hadoop.ozone.container.ozoneimpl.ContainerReader.run(ContainerReader.java:108)
at java.lang.Thread.run(Thread.java:748)
2021-02-27 18:22:50,466 [Thread-3] INFO volume.MutableVolumeSet: Moving Volume : /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds to failed Volumes
2021-02-27 18:22:50,466 [main] INFO ozoneimpl.OzoneContainer: Build ContainerSet costs 0s
Failed Volume:/tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds

Output while running the conversion tool

➜ ~/code/apache/ozone/oz_scm_ha/hadoop-ozone/dist/target/ozone-1.1.0-SNAPSHOT git:(HDDS-4866) bin/ozone debug dnlayout --path=/tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers
2021-02-27 18:23:05,042 [main] INFO fs.SaveSpaceUsageToFile: Cached usage info file /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/scmUsed not found
2021-02-27 18:23:05,045 [main] INFO volume.HddsVolume: Creating Volume: /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds of storage type : DISK and capacity : 499963174912
2021-02-27 18:23:05,047 [main] INFO volume.MutableVolumeSet: Added Volume : /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds to VolumeSet
2021-02-27 18:23:05,053 [main] INFO volume.ThrottledAsyncChecker: Scheduling a check for /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:23:05,065 [main] INFO volume.HddsVolumeChecker: Scheduled health check for volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:23:05,089 [main] INFO ozoneimpl.ContainerReader: Running in upgrade mode:true
2021-02-27 18:23:05,090 [Thread-3] INFO ozoneimpl.ContainerReader: Start to verify containers on volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:23:05,090 [Thread-3] INFO ozoneimpl.ContainerReader: Storage dir based on clusterId doesn't exists.Renaming storage location:/tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds/7af268c4-5139-40f3-a199-8b0d77a62581 to /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds/510b80d0-fd9a-42f9-ae9a-7828893aedac
2021-02-27 18:23:05,833 [Thread-3] INFO ozoneimpl.ContainerReader: Finish verifying containers on volume /tmp/chaos-2021-02-26-11-05-48-IST/MiniOzoneClusterImpl-510b80d0-fd9a-42f9-ae9a-7828893aedac/datanode-3/data-1/containers/hdds
2021-02-27 18:23:05,833 [main] INFO ozoneimpl.OzoneContainer: Build ContainerSet costs 0s

bshashikant
bshashikant requested changes Feb 28, 2021
View reviewed changes
...ainer-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java Show resolved Hide resolved
...ainer-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java Show resolved Hide resolved
...ainer-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java Outdated Show resolved Hide resolved
...ainer-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java Show resolved Hide resolved
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DatanodeLayout.java Outdated Show resolved Hide resolved
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DatanodeLayout.java Show resolved Hide resolved
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/DatanodeLayout.java Show resolved Hide resolved
@mukul1987 mukul1987 force-pushed the HDDS-4866 branch 3 times, most recently from 770fd0b to 066e794 Compare March 2, 2021 11:52
bshashikant
bshashikant reviewed Mar 3, 2021
View reviewed changes
Copy link
Contributor

@bshashikant bshashikant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good apart from few CI failures.

bshashikant
bshashikant approved these changes Mar 4, 2021
View reviewed changes
Copy link
Contributor

@bshashikant bshashikant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mukul1987 for the contribution.

@bshashikant bshashikant merged commit 6c8fea8 into apache:HDDS-2823 Mar 4, 2021
@mukul1987 mukul1987 deleted the HDDS-4866 branch March 4, 2021 07:41
mukul1987 added a commit to mukul1987/hadoop-ozone that referenced this pull request Mar 12, 2021
@mukul1987
HDDS-4866. Datanode with scmID format should work with clusterID dire… (
abff311 
apache#1959)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bshashikant bshashikant bshashikant approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@mukul1987 @bshashikant