Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-2943. Parameterize unit tests for chunk manager implementation #694

Merged
merged 2 commits into from Mar 31, 2020

Conversation

adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

  1. Apply some unit tests for both kinds of chunk layout (per-chunk, per-block).
  2. Fix a bug in container data scrubber, which was still using per-chunk layout regardless of config. This can result in marking containers unhealthy:
[ContainerDataScanner(/data/hdds/hdds)] ERROR interfaces.Container: Corruption detected in container: [1] Exception: [Missing chunk file /data/hdds/hdds/1560ebb4-2052-44ee-b16e-d74caaca055e/current/containerDir0/1/chunks/103843356633071616_chunk_1]
[ContainerDataScanner(/data/hdds/hdds)] INFO ozoneimpl.ContainerDataScanner: Completed an iteration of container data scrubber in 0 minutes. Number of iterations (since the data-node restart) : 1, Number of containers scanned in this iteration : 1, Number of unhealthy containers found in this iteration : 1

https://issues.apache.org/jira/browse/HDDS-2943

How was this patch tested?

https://github.com/adoroszlai/hadoop-ozone/runs/516276869

Verified that data scrubber does not mark container as unhealthy:

$ ozone freon ockg -n 1 -p test
$ ozone scmcli container close 1
$ ozone freon ockv -n 1 -p test
...
Successful executions: 1
[ContainerDataScanner(/data/hdds/hdds)] INFO ozoneimpl.ContainerDataScanner: Completed an iteration of container data scrubber in 0 minutes. Number of iterations (since the data-node restart) : 1, Number of containers scanned in this iteration : 1, Number of unhealthy containers found in this iteration : 0

@adoroszlai adoroszlai self-assigned this Mar 18, 2020
@bshashikant bshashikant merged commit 33ac261 into apache:master Mar 31, 2020
@adoroszlai adoroszlai deleted the HDDS-2943 branch March 31, 2020 07:41
@adoroszlai
Copy link
Contributor Author

Thanks @bshashikant for reviewing and merging it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants