Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-18306: Warnings should not be shown on cli console when linux user not present on client #4474

Merged
merged 2 commits into from
Jun 28, 2022
Merged

HADOOP-18306: Warnings should not be shown on cli console when linux user not present on client #4474

merged 2 commits into from
Jun 28, 2022

Conversation

swamirishi
Copy link
Contributor

Description of PR

Warnings should not be shown on cli console when linux user not present on client

How was this patch tested?

Logging Changes

umamaheswararao
umamaheswararao reviewed Jun 20, 2022
View reviewed changes
...ject/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedUnixGroupsMapping.java Outdated
@@ -215,7 +215,13 @@ private Set<String> getUnixGroups(String user) throws IOException {
groups = resolvePartialGroupNames(user, e.getMessage(),
executor.getOutput());
} catch (PartialGroupNameException pge) {
LOG.warn("unable to return groups for user {}", user, pge);
if(LOG.isDebugEnabled()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just extract this into String msg = ""unable to return groups for user {}" and use this msg to make log line looks clean?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smengcl do you think this is fine? or we don't need this in even in Warn?
we were hitting this log message. This current patch voids the full trace in logs when debug is not enabled.

Copy link
Contributor

@smengcl smengcl Jun 20, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes IMO we don't need the warn.

@swamirishi Just be sure to double check the debug log can actually be enabled for example by setting org.apache.hadoop.security.ShellBasedUnixGroupsMapping=DEBUG in log4j.properties or JVM -D param.

nit: checkstyle missing space after if

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@swamirishi could you please fix the @smengcl comments above? you need to format line number 218

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is stale version. Thanks

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 45s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 37m 14s trunk passed
+1 💚 compile 22m 58s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 20m 34s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 47s trunk passed
+1 💚 mvnsite 2m 13s trunk passed
+1 💚 javadoc 1m 50s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 16s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 3m 12s trunk passed
+1 💚 shadedclient 23m 34s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 8s the patch passed
+1 💚 compile 22m 13s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 22m 13s the patch passed
+1 💚 compile 20m 37s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 20m 37s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 41s the patch passed
+1 💚 mvnsite 2m 10s the patch passed
+1 💚 javadoc 1m 38s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 21s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 3m 9s the patch passed
+1 💚 shadedclient 23m 36s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 18m 29s hadoop-common in the patch passed.
+1 💚 asflicense 1m 36s The patch does not generate ASF License warnings.
214m 32s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/1/artifact/out/Dockerfile
GITHUB PR #4474
JIRA Issue HADOOP-18306
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a88b3d64dc08 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 26b3148
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/1/testReport/
Max. process+thread count 3158 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 55s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 24s trunk passed
+1 💚 compile 25m 10s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 21m 37s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 1m 31s trunk passed
+1 💚 mvnsite 1m 58s trunk passed
+1 💚 javadoc 1m 31s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 4s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 3m 6s trunk passed
+1 💚 shadedclient 25m 52s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 6s the patch passed
+1 💚 compile 24m 13s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 24m 13s the patch passed
+1 💚 compile 21m 45s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 21m 45s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 24s the patch passed
+1 💚 mvnsite 1m 56s the patch passed
+1 💚 javadoc 1m 22s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 4s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 3m 3s the patch passed
+1 💚 shadedclient 25m 37s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 18m 17s hadoop-common in the patch passed.
+1 💚 asflicense 1m 16s The patch does not generate ASF License warnings.
224m 51s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/2/artifact/out/Dockerfile
GITHUB PR #4474
JIRA Issue HADOOP-18306
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 472d91b4539c 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 370df9b
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/2/testReport/
Max. process+thread count 1803 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4474/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@umamaheswararao
Copy link
Contributor

The -1 is due to no tests added.

-1 overall

[2022-06-23T22:22:07.359Z]

[2022-06-23T22:22:07.359Z] | Vote | Subsystem | Runtime | Comment

[2022-06-23T22:22:07.359Z] ============================================================================

[2022-06-23T22:22:07.360Z] | 0 | reexec | 0m 55s | Docker mode activated.

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | | | | Prechecks

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | +1 | dupname | 0m 0s | No case conflicting files found.

[2022-06-23T22:22:07.360Z] | 0 | codespell | 0m 0s | codespell was not available.

[2022-06-23T22:22:07.360Z] | 0 | detsecrets | 0m 0s | detect-secrets was not available.

[2022-06-23T22:22:07.360Z] | +1 | @author | 0m 0s | The patch does not contain any @author

[2022-06-23T22:22:07.360Z] | | | | tags.

[2022-06-23T22:22:07.360Z] | -1 | test4tests | 0m 0s | The patch doesn't appear to include

[2022-06-23T22:22:07.360Z] | | | | any new or modified tests. Please

[2022-06-23T22:22:07.360Z] | | | | justify why no new tests are needed for

[2022-06-23T22:22:07.360Z] | | | | this patch. Also please list what

[2022-06-23T22:22:07.360Z] | | | | manual steps were performed to verify

[2022-06-23T22:22:07.360Z] | | | | this patch.

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | | | | trunk Compile Tests

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | +1 | mvninstall | 40m 24s | trunk passed

[2022-06-23T22:22:07.360Z] | +1 | compile | 25m 10s | trunk passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

[2022-06-23T22:22:07.360Z] | | | |

[2022-06-23T22:22:07.360Z] | +1 | compile | 21m 37s | trunk passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04

[2022-06-23T22:22:07.360Z] | | | | -b07

[2022-06-23T22:22:07.360Z] | +1 | checkstyle | 1m 31s | trunk passed

[2022-06-23T22:22:07.360Z] | +1 | mvnsite | 1m 58s | trunk passed

[2022-06-23T22:22:07.360Z] | +1 | javadoc | 1m 31s | trunk passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

[2022-06-23T22:22:07.360Z] | | | |

[2022-06-23T22:22:07.360Z] | +1 | javadoc | 1m 4s | trunk passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04

[2022-06-23T22:22:07.360Z] | | | | -b07

[2022-06-23T22:22:07.360Z] | +1 | spotbugs | 3m 6s | trunk passed

[2022-06-23T22:22:07.360Z] | +1 | shadedclient | 25m 52s | branch has no errors when building and

[2022-06-23T22:22:07.360Z] | | | | testing our client artifacts.

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | | | | Patch Compile Tests

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | +1 | mvninstall | 1m 6s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | compile | 24m 13s | the patch passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

[2022-06-23T22:22:07.360Z] | | | |

[2022-06-23T22:22:07.360Z] | +1 | javac | 24m 13s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | compile | 21m 45s | the patch passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04

[2022-06-23T22:22:07.360Z] | | | | -b07

[2022-06-23T22:22:07.360Z] | +1 | javac | 21m 45s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | blanks | 0m 0s | The patch has no blanks issues.

[2022-06-23T22:22:07.360Z] | +1 | checkstyle | 1m 24s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | mvnsite | 1m 56s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | javadoc | 1m 22s | the patch passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1

[2022-06-23T22:22:07.360Z] | | | |

[2022-06-23T22:22:07.360Z] | +1 | javadoc | 1m 4s | the patch passed with JDK Private

[2022-06-23T22:22:07.360Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04

[2022-06-23T22:22:07.360Z] | | | | -b07

[2022-06-23T22:22:07.360Z] | +1 | spotbugs | 3m 3s | the patch passed

[2022-06-23T22:22:07.360Z] | +1 | shadedclient | 25m 37s | patch has no errors when building and

[2022-06-23T22:22:07.360Z] | | | | testing our client artifacts.

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | | | | Other Tests

[2022-06-23T22:22:07.360Z] +---------------------------------------------------------------------------

[2022-06-23T22:22:07.360Z] | +1 | unit | 18m 17s | hadoop-common in the patch passed.

[2022-06-23T22:22:07.360Z] | +1 | asflicense | 1m 16s | The patch does not generate ASF

[2022-06-23T22:22:07.360Z] | | | | License warnings.

[2022-06-23T22:22:07.360Z] | | | 224m 51s |

Since this is only log message level change, I am approving this change.

umamaheswararao
umamaheswararao approved these changes Jun 28, 2022
View reviewed changes
Copy link
Contributor

@umamaheswararao umamaheswararao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@umamaheswararao umamaheswararao merged commit 43112bd into apache:trunk Jun 28, 2022
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@swamirishi
HADOOP-18306: Warnings should not be shown on cli console when linux …
da1060c 
…user not present on client (apache#4474). Contributed by swamirishi.
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
@swamirishi
CDPD-39540. HADOOP-18306: Warnings should not be shown on cli console…
d22270e 
… when linux user not present on client (apache#4474). Contributed by swamirishi.

Change-Id: Ia512ab790f501fb2b7f52773c2d4a8d8692d9d1d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smengcl smengcl smengcl left review comments

@umamaheswararao umamaheswararao umamaheswararao approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@swamirishi @hadoop-yetus @umamaheswararao @smengcl