Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TEZ-4340: Show convenient input -> output vertex names in input messages #170

Merged
merged 1 commit into from
Jan 17, 2022
Merged

TEZ-4340: Show convenient input -> output vertex names in input messages #170

merged 1 commit into from
Jan 17, 2022

Conversation

csjuhasz-c
Copy link
Contributor

No description provided.

@tez-yetus

This comment has been minimized.

Sign in to view
@abstractdog
Copy link
Contributor

abstractdog commented Dec 25, 2021 •
edited

patch looks good mostly, some comments:

  1. please replace srcNameTrimmed in ShuffleManager and ShuffleScheduler too, similarly to UnorderedPartitionedKVWriter in TEZ-4180
  2. address checkstyle/whitespace warnings where possible

@tez-yetus tez-yetus mentioned this pull request Dec 26, 2021
Merged
@tez-yetus

This comment has been minimized.

Sign in to view
@tez-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 4 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 4m 47s Maven dependency ordering for branch
+1 💚 mvninstall 10m 17s master passed
+1 💚 compile 2m 43s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 compile 2m 40s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 2m 23s master passed
+1 💚 javadoc 2m 46s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 2m 14s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗 spotbugs 0m 46s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 4m 37s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 9s Maven dependency ordering for patch
+1 💚 mvninstall 1m 44s the patch passed
+1 💚 compile 1m 49s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javac 1m 49s the patch passed
+1 💚 compile 1m 36s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 1m 36s the patch passed
+1 💚 checkstyle 0m 13s The patch passed checkstyle in tez-api
+1 💚 checkstyle 0m 9s The patch passed checkstyle in tez-runtime-internals
-0 ⚠️ checkstyle 0m 19s tez-runtime-library: The patch generated 4 new + 160 unchanged - 3 fixed = 164 total (was 163)
+1 💚 checkstyle 0m 12s The patch passed checkstyle in tez-mapreduce
+1 💚 checkstyle 0m 13s tez-tests: The patch generated 0 new + 39 unchanged - 1 fixed = 39 total (was 40)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 1m 31s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 24s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 findbugs 4m 7s the patch passed
_ Other Tests _
+1 💚 unit 2m 2s tez-api in the patch passed.
+1 💚 unit 0m 35s tez-runtime-internals in the patch passed.
+1 💚 unit 4m 18s tez-runtime-library in the patch passed.
+1 💚 unit 1m 14s tez-mapreduce in the patch passed.
+1 💚 unit 46m 29s tez-tests in the patch passed.
+1 💚 asflicense 0m 57s The patch does not generate ASF License warnings.
104m 5s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/3/artifact/out/Dockerfile
GITHUB PR #170
JIRA Issue TEZ-4340
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux 8f243b6da851 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 41cbc17
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
checkstyle https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/3/artifact/out/diff-checkstyle-tez-runtime-library.txt
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/3/testReport/
Max. process+thread count 1398 (vs. ulimit of 5500)
modules C: tez-api tez-runtime-internals tez-runtime-library tez-mapreduce tez-tests U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/3/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@tez-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 4m 48s Maven dependency ordering for branch
+1 💚 mvninstall 9m 33s master passed
+1 💚 compile 2m 27s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 compile 2m 14s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 2m 3s master passed
+1 💚 javadoc 2m 22s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 2m 2s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗 spotbugs 0m 44s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 4m 26s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 1m 35s the patch passed
+1 💚 compile 1m 41s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javac 1m 41s the patch passed
+1 💚 compile 1m 29s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 1m 29s the patch passed
+1 💚 checkstyle 0m 11s The patch passed checkstyle in tez-api
+1 💚 checkstyle 0m 9s The patch passed checkstyle in tez-runtime-internals
-0 ⚠️ checkstyle 0m 18s tez-runtime-library: The patch generated 7 new + 274 unchanged - 4 fixed = 281 total (was 278)
+1 💚 checkstyle 0m 11s The patch passed checkstyle in tez-mapreduce
+1 💚 checkstyle 0m 12s tez-tests: The patch generated 0 new + 39 unchanged - 1 fixed = 39 total (was 40)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 1m 25s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 17s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 findbugs 3m 59s the patch passed
_ Other Tests _
+1 💚 unit 1m 58s tez-api in the patch passed.
+1 💚 unit 0m 32s tez-runtime-internals in the patch passed.
+1 💚 unit 4m 5s tez-runtime-library in the patch passed.
+1 💚 unit 1m 11s tez-mapreduce in the patch passed.
+1 💚 unit 43m 30s tez-tests in the patch passed.
+1 💚 asflicense 0m 49s The patch does not generate ASF License warnings.
97m 42s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/4/artifact/out/Dockerfile
GITHUB PR #170
JIRA Issue TEZ-4340
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux 7fbff432f002 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 41cbc17
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
checkstyle https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/4/artifact/out/diff-checkstyle-tez-runtime-library.txt
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/4/testReport/
Max. process+thread count 2089 (vs. ulimit of 5500)
modules C: tez-api tez-runtime-internals tez-runtime-library tez-mapreduce tez-tests U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/4/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@tez-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 6 new or modified test files.
_ master Compile Tests _
+0 🆗 mvndep 4m 41s Maven dependency ordering for branch
+1 💚 mvninstall 9m 8s master passed
+1 💚 compile 2m 44s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 compile 2m 41s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 2m 33s master passed
+1 💚 javadoc 2m 48s master passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 2m 27s master passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗 spotbugs 0m 46s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 4m 37s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 1m 33s the patch passed
+1 💚 compile 1m 36s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javac 1m 36s the patch passed
+1 💚 compile 1m 28s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 1m 28s the patch passed
+1 💚 checkstyle 0m 12s The patch passed checkstyle in tez-api
+1 💚 checkstyle 0m 10s The patch passed checkstyle in tez-runtime-internals
-0 ⚠️ checkstyle 0m 21s tez-runtime-library: The patch generated 4 new + 274 unchanged - 4 fixed = 278 total (was 278)
+1 💚 checkstyle 0m 12s The patch passed checkstyle in tez-mapreduce
+1 💚 checkstyle 0m 12s tez-tests: The patch generated 0 new + 39 unchanged - 1 fixed = 39 total (was 40)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 1m 29s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 19s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 findbugs 3m 49s the patch passed
_ Other Tests _
+1 💚 unit 2m 1s tez-api in the patch passed.
+1 💚 unit 0m 35s tez-runtime-internals in the patch passed.
+1 💚 unit 4m 28s tez-runtime-library in the patch passed.
+1 💚 unit 1m 13s tez-mapreduce in the patch passed.
+1 💚 unit 48m 26s tez-tests in the patch passed.
+1 💚 asflicense 0m 58s The patch does not generate ASF License warnings.
104m 35s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/5/artifact/out/Dockerfile
GITHUB PR #170
JIRA Issue TEZ-4340
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux 0607e7c8856b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 41cbc17
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
checkstyle https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/5/artifact/out/diff-checkstyle-tez-runtime-library.txt
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/5/testReport/
Max. process+thread count 1362 (vs. ulimit of 5500)
modules C: tez-api tez-runtime-internals tez-runtime-library tez-mapreduce tez-tests U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-170/5/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

abstractdog
abstractdog approved these changes Jan 16, 2022
View reviewed changes
Copy link
Contributor

@abstractdog abstractdog left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@abstractdog abstractdog merged commit cc8249e into apache:master Jan 17, 2022
This was referenced Feb 25, 2022
Merged
Merged
@tez-yetus tez-yetus mentioned this pull request Mar 30, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abstractdog abstractdog abstractdog approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@csjuhasz-c @tez-yetus @abstractdog