Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-3284. ozonesecure-mr test fails due to lack of disk space #725

Merged
merged 2 commits into from
Mar 26, 2020

Conversation

adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Disable YARN disk utilization check in ozonesecure-mr acceptance test.

Plenty of disk space is available in CI, but more than 90% of the disk is used:

Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sda1        84G   75G  8.3G  91% /

Thus directory checker marks it as invalid:

WARN  DirectoryCollection:418 - Directory /tmp/hadoop-hadoop/nm-local-dir error, used space above threshold of 90.0%, removing from list of valid directories
WARN  DirectoryCollection:418 - Directory /opt/hadoop/logs/userlogs error, used space above threshold of 90.0%, removing from list of valid directories

https://issues.apache.org/jira/browse/HDDS-3284

How was this patch tested?

https://github.com/adoroszlai/hadoop-ozone/runs/535925282

==============================================================================
Execute PI calculation                                                | PASS |
------------------------------------------------------------------------------
Execute WordCount                                                     | PASS |
------------------------------------------------------------------------------
ozonesecure-mr-mapreduce :: Execute MR jobs                           | PASS |

@adoroszlai adoroszlai self-assigned this Mar 26, 2020
@sodonnel
Copy link
Contributor

Thanks for looking into this. +1 on this from me, pending CI, but some of the "it-*" tests are a bit flaky.

@adoroszlai
Copy link
Contributor Author

Thanks @sodonnel. Since it only changes acceptance tests, I think we can merge it even if it-client happens to fail.

@sodonnel
Copy link
Contributor

Yea, I have seen random failures on it-client. As this is a yarn only change, I cannot see how it could impact the integration tests, so I think we are good to merge it.

@vivekratnavel
Copy link
Contributor

+1 LGTM.

@hanishakoneru
Copy link
Contributor

Thanks @adoroszlai for working on this. I will go ahead and merge this as the change is only in acceptance tests.
Thanks @sodonnel and @vivekratnavel for the reviews.

@hanishakoneru hanishakoneru merged commit 07fcb79 into apache:master Mar 26, 2020
@adoroszlai
Copy link
Contributor Author

Thanks @sodonnel and @vivekratnavel for the review, and @hanishakoneru for merging it.

@adoroszlai adoroszlai deleted the HDDS-3284 branch March 26, 2020 20:50
isahkemat pushed a commit to isahkemat/hadoop-ozone that referenced this pull request Mar 29, 2020
adoroszlai added a commit to adoroszlai/ozone that referenced this pull request Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants