Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YARN-9655:AllocateResponse in FederationInterceptor lost applicationPriority #1023

Closed
wants to merge 2 commits into from

Conversation

@hunshenshi
Copy link
Contributor

hunshenshi commented Jun 27, 2019

In YARN Federation mode using FederationInterceptor, when submitting application, am will report an error.

2019-06-25 11:44:00,977 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: ERROR IN CONTACTING RM. java.lang.NullPointerException 
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.handleJobPriorityChange(RMContainerAllocator.java:1025) 
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.getResources(RMContainerAllocator.java:880) 
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:286) 
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$AllocatorRunnable.run(RMCommunicator.java:280) 
at java.lang.Thread.run(Thread.java:748)

The reason is that applicationPriority is lost.

@hadoop-yetus

This comment has been minimized.

Copy link

hadoop-yetus commented Jun 27, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 119 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1284 trunk passed
+1 compile 67 trunk passed
+1 checkstyle 25 trunk passed
+1 mvnsite 42 trunk passed
+1 shadedclient 721 branch has no errors when building and testing our client artifacts.
+1 javadoc 26 trunk passed
0 spotbugs 87 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 85 trunk passed
_ Patch Compile Tests _
+1 mvninstall 39 the patch passed
+1 compile 66 the patch passed
+1 javac 66 the patch passed
-0 checkstyle 21 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 mvnsite 41 the patch passed
+1 whitespace 1 The patch has no whitespace issues.
+1 shadedclient 757 patch has no errors when building and testing our client artifacts.
+1 javadoc 25 the patch passed
+1 findbugs 87 the patch passed
_ Other Tests _
+1 unit 1365 hadoop-yarn-server-nodemanager in the patch passed.
+1 asflicense 32 The patch does not generate ASF License warnings.
4892
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/1/artifact/out/Dockerfile
GITHUB PR #1023
JIRA Issue YARN-9655
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 0bef0840670c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / be80334
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/1/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/1/testReport/
Max. process+thread count 444 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

This comment has been minimized.

Copy link

hadoop-yetus commented Jun 28, 2019

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 53 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 1 The patch does not contain any @author tags.
-1 test4tests 0 The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 mvninstall 1069 trunk passed
+1 compile 67 trunk passed
+1 checkstyle 27 trunk passed
+1 mvnsite 43 trunk passed
+1 shadedclient 762 branch has no errors when building and testing our client artifacts.
+1 javadoc 29 trunk passed
0 spotbugs 78 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 75 trunk passed
_ Patch Compile Tests _
+1 mvninstall 36 the patch passed
+1 compile 59 the patch passed
+1 javac 59 the patch passed
+1 checkstyle 23 the patch passed
+1 mvnsite 37 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 756 patch has no errors when building and testing our client artifacts.
+1 javadoc 22 the patch passed
+1 findbugs 77 the patch passed
_ Other Tests _
+1 unit 1313 hadoop-yarn-server-nodemanager in the patch passed.
+1 asflicense 32 The patch does not generate ASF License warnings.
4564
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/2/artifact/out/Dockerfile
GITHUB PR #1023
JIRA Issue YARN-9655
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux adc7662a227d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 4a21224
Default Java 1.8.0_212
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/2/testReport/
Max. process+thread count 446 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-1023/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@asfgit asfgit closed this in 5e7caf1 Jun 28, 2019
asfgit pushed a commit that referenced this pull request Jun 28, 2019
…licationPriority. (hunshenshi via wwei) closes #1023"

This reverts commit 5e7caf1.
bshashikant added a commit to bshashikant/hadoop that referenced this pull request Jul 10, 2019
bshashikant added a commit to bshashikant/hadoop that referenced this pull request Jul 10, 2019
…licationPriority. (hunshenshi via wwei) closes apache#1023"

This reverts commit 5e7caf1.
arp7 pushed a commit to arp7/hadoop that referenced this pull request Sep 18, 2019
…Priority. (hunshenshi via wwei) closes apache#1023

(cherry picked from commit 5e7caf1)

Change-Id: Iad1e39e792bee0bfe308cfeaf75f85ee196e2f29
arp7 pushed a commit to arp7/hadoop that referenced this pull request Sep 18, 2019
…licationPriority. (hunshenshi via wwei) closes apache#1023"

This reverts commit 5e7caf1.

(cherry picked from commit f09c31a)

Change-Id: I0c1f8722343c8c6417a7c15dcf973d15f891601b
shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
SAMZA-2183: print all tasks when job fails with incorrect container count
amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019
…licationPriority. (hunshenshi via wwei) closes apache#1023"

This reverts commit 5e7caf1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.