Skip to content

Comments

YARN-10399 Refactor NodeQueueLoadMonitor class to make it extendable#2228

Merged
goiri merged 3 commits intoapache:trunkfrom
zhengbli:YARN-10399
Aug 19, 2020
Merged

YARN-10399 Refactor NodeQueueLoadMonitor class to make it extendable#2228
goiri merged 3 commits intoapache:trunkfrom
zhengbli:YARN-10399

Conversation

@zhengbli
Copy link
Contributor

No description provided.

@goiri
Copy link
Member

goiri commented Aug 14, 2020

I just have a comment about using logger as we are moving code around but I'm fine without changing that.
I'll wait for Yetus to run the tests and confirm but overall this is a clean refactor.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 28m 47s trunk passed
+1 💚 compile 0m 57s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 compile 0m 51s trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 checkstyle 0m 42s trunk passed
+1 💚 mvnsite 0m 55s trunk passed
+1 💚 shadedclient 15m 34s branch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 40s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javadoc 0m 37s trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗 spotbugs 1m 43s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 1m 41s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 48s the patch passed
+1 💚 compile 0m 49s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javac 0m 49s the patch passed
+1 💚 compile 0m 43s the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 javac 0m 43s the patch passed
-0 ⚠️ checkstyle 0m 31s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 15 new + 2 unchanged - 2 fixed = 17 total (was 4)
+1 💚 mvnsite 0m 45s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 14m 14s patch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 32s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javadoc 0m 32s the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 findbugs 1m 46s the patch passed
_ Other Tests _
-1 ❌ unit 94m 27s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
168m 33s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/1/artifact/out/Dockerfile
GITHUB PR #2228
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux c16aad18a6c4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / b93dd7c
Default Java Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
checkstyle https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/1/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
unit https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/1/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/1/testReport/
Max. process+thread count 868 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/1/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@BilwaST
Copy link
Contributor

BilwaST commented Aug 18, 2020

Thanks @zhengbli for patch. Overall patch looks good me. Just a minor nit.
can you add a comment in below code which you removed?
if (!excludeFullNodes || !cNode.isQueueFull()) { retList.add(cNode);

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 12s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 31m 28s trunk passed
+1 💚 compile 1m 9s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 compile 0m 54s trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 checkstyle 0m 42s trunk passed
+1 💚 mvnsite 0m 58s trunk passed
+1 💚 shadedclient 15m 40s branch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 40s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javadoc 0m 36s trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+0 🆗 spotbugs 1m 45s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 1m 42s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 46s the patch passed
+1 💚 compile 0m 49s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javac 0m 49s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 javac 0m 41s the patch passed
-0 ⚠️ checkstyle 0m 30s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 16 new + 1 unchanged - 3 fixed = 17 total (was 4)
+1 💚 mvnsite 0m 46s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 13m 55s patch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 35s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javadoc 0m 33s the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
+1 💚 findbugs 1m 42s the patch passed
_ Other Tests _
-1 ❌ unit 92m 12s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 33s The patch does not generate ASF License warnings.
169m 35s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/2/artifact/out/Dockerfile
GITHUB PR #2228
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 4afdf8e4fc64 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b65e43f
Default Java Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01
checkstyle https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/2/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
unit https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/2/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/2/testReport/
Max. process+thread count 864 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2228/2/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@goiri
Copy link
Member

goiri commented Aug 19, 2020

@BilwaST do you have further comments?

@BilwaST
Copy link
Contributor

BilwaST commented Aug 19, 2020

@BilwaST do you have further comments?

@goiri no..you can commit this.

@goiri goiri merged commit 9b9f7ea into apache:trunk Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants