Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-18219. Revert "HADOOP-15983. Use jersey-json that is built to use jackson2 (#3988)" #4253

Closed
wants to merge 2 commits into from

Conversation

GauthamBanasandra
Copy link
Member

Description of PR

This reverts commit 6318708. This commit seems to break the shadedclient tests on Debian 10 - #4245 (comment).

How was this patch tested?

In progress.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@GauthamBanasandra
Copy link
Member Author

Hi @pjfanning, @aajisaka, @steveloughran,

This commit has broken the shadedclient tests on Debian 10 - 6318708. The last known good commit where shadedclient tests ran successfully is aebd55f.

Here are the steps that I followed -

  1. Create docker image for building Hadoop on Debian 10 using /dev-support/docker/Dockerfile_debian_10 -
$ docker build -t hadoop-builder-debian-10 -f .\dev-support\docker\Dockerfile_debian_10 .\dev-support\docker\
  1. Start the container from the image hadoop-builder-debian-10 that we just created -
$ docker run -it hadoop-builder-debian-10 bash
  1. Clone the Hadoop repo -
$ git clone https://github.com/apache/hadoop.git
$ cd hadoop
  1. Start git bisect -
$ git bisect start
$ git bisect good aebd55f7883c6b12afe5faeb776ab0e0b83420da
$ git bisect bad 63187083cc3b9bb1c1e90e692e271958561f9cc8
  1. Run the following command every time git bisect checks out a commit -
$ /usr/bin/mvn verify -fae -am -pl hadoop-client-modules/hadoop-client-check-invariants -pl hadoop-client-modules/hadoop-client-check-test-invariants -pl hadoop-client-modules/hadoop-client-integration-tests -Dtest=NoUnitTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dspotbugs.skip=true

@pjfanning
Copy link
Contributor

If you think reverting the changes altogether is best, then feel free to do this. The PR you are reverting had pom changes for hadoop-client-minicluster including exclusions of the jaxb transitive dependencies - probably due to dependency convergence errors. Adding explicit dependencies to the jaxb jars in hadoop-client-minicluster might fix the issue. I'm currently travelling and am not in an ideal position to try any changes myself (apologies).

@aajisaka
Copy link
Member

aajisaka commented May 1, 2022

Thanks @GauthamBanasandra for your report and thanks @pjfanning for your comment. I opened #4254 to fix the exclusion. Would you check the PR?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 45m 24s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 shellcheck 0m 1s Shellcheck was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 15m 39s Maven dependency ordering for branch
+1 💚 mvninstall 28m 39s trunk passed
+1 💚 compile 21m 29s trunk passed
+1 💚 mvnsite 20m 20s trunk passed
+1 💚 javadoc 8m 53s trunk passed
+1 💚 shadedclient 117m 34s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 44s Maven dependency ordering for patch
+1 💚 mvninstall 38m 40s the patch passed
+1 💚 compile 21m 3s the patch passed
-1 ❌ cc 21m 3s /results-compile-cc-root.txt root generated 6 new + 199 unchanged - 6 fixed = 205 total (was 205)
+1 💚 golang 21m 3s the patch passed
+1 💚 javac 21m 3s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 19m 55s the patch passed
+1 💚 xml 0m 15s The patch has no ill-formed XML file.
+1 💚 javadoc 8m 44s the patch passed
+1 💚 shadedclient 32m 4s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 1014m 36s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 2m 9s The patch does not generate ASF License warnings.
1294m 12s
Reason Tests
Failed junit tests hadoop.hdfs.server.federation.security.TestRouterSecurityManager
hadoop.yarn.sls.TestSLSRunner
hadoop.yarn.sls.TestSLSStreamAMSynth
hadoop.yarn.sls.TestSLSDagAMSimulator
hadoop.yarn.sls.nodemanager.TestNMSimulator
hadoop.yarn.sls.TestSLSGenericSynth
hadoop.yarn.sls.TestReservationSystemInvariants
hadoop.yarn.sls.appmaster.TestAMSimulator
hadoop.yarn.client.TestGetGroups
hadoop.yarn.client.TestResourceManagerAdministrationProtocolPBClientImpl
hadoop.yarn.server.router.webapp.TestRouterWebServicesREST
hadoop.yarn.server.timeline.webapp.TestTimelineWebServicesWithSSL
hadoop.yarn.server.timeline.security.TestTimelineAuthenticationFilterForV1
hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
hadoop.yarn.server.resourcemanager.TestRMHA
hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher
hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher
hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
hadoop.yarn.webapp.TestRMWithXFSFilter
hadoop.yarn.server.resourcemanager.TestClientRMService
hadoop.mapred.TestLocalDistributedCacheManager
hadoop.crypto.TestCryptoCodec
hadoop.service.launcher.TestServiceInterruptHandling
hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
hadoop.crypto.key.kms.server.TestKMSWithZK
hadoop.crypto.key.kms.server.TestKMS
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4253/1/artifact/out/Dockerfile
GITHUB PR #4253
Optional Tests dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml cc golang
uname Linux d2c35765bc31 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2c00f2b
Default Java Red Hat, Inc.-1.8.0_322-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4253/1/testReport/
Max. process+thread count 3137 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp hadoop-client-modules/hadoop-client hadoop-tools/hadoop-resourceestimator hadoop-client-modules/hadoop-client-minicluster . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4253/1/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@GauthamBanasandra
Copy link
Member Author

Abandoning this PR since the issue was fixed by #4254.

@GauthamBanasandra GauthamBanasandra deleted the revert-jj branch May 3, 2022 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants