Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS-16604. Install gtest via FetchContent_Declare in CMake #4374

Merged
merged 7 commits into from
Jun 1, 2022

Conversation

GauthamBanasandra
Copy link
Member

@GauthamBanasandra GauthamBanasandra commented May 29, 2022

Description of PR

CMake is unable to checkout release-1.10.0 version of GoogleTest -

[WARNING] -- Build files have been written to: /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4370/centos-7/src/hadoop-hdfs-project/hadoop-hdfs-native-client/target/main/native/libhdfspp/googletest-download
[WARNING] Scanning dependencies of target googletest
[WARNING] [ 11%] Creating directories for 'googletest'
[WARNING] [ 22%] Performing download step (git clone) for 'googletest'
[WARNING] Cloning into 'googletest-src'...
[WARNING] fatal: invalid reference: release-1.10.0
[WARNING] CMake Error at googletest-download/googletest-prefix/tmp/googletest-gitclone.cmake:40 (message):
[WARNING]   Failed to checkout tag: 'release-1.10.0'
[WARNING] 
[WARNING] 
[WARNING] gmake[2]: *** [CMakeFiles/googletest.dir/build.make:111: googletest-prefix/src/googletest-stamp/googletest-download] Error 1
[WARNING] gmake[1]: *** [CMakeFiles/Makefile2:95: CMakeFiles/googletest.dir/all] Error 2
[WARNING] gmake: *** [Makefile:103: all] Error 2
[WARNING] CMake Error at main/native/libhdfspp/CMakeLists.txt:68 (message):
[WARNING]   Build step for googletest failed: 2

Jenkins run - https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4370/6/artifact/out/branch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt

We need to use FetchContent_Declare since we're getting the source code exactly at the given commit SHA. This avoids the checkout step altogether and solves the above issue.

How was this patch tested?

Hadoop Jenkins CI run.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus

This comment was marked as outdated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 16s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 37s trunk passed
-1 ❌ compile 0m 36s /branch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt hadoop-hdfs-native-client in trunk failed.
+1 💚 mvnsite 0m 38s trunk passed
+1 💚 shadedclient 64m 5s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 22s the patch passed
+1 💚 compile 3m 48s the patch passed
-1 ❌ cc 3m 48s /results-compile-cc-hadoop-hdfs-project_hadoop-hdfs-native-client.txt hadoop-hdfs-project_hadoop-hdfs-native-client generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 💚 golang 3m 48s the patch passed
+1 💚 javac 3m 48s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 23s the patch passed
+1 💚 shadedclient 22m 9s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 109m 16s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 47s The patch does not generate ASF License warnings.
204m 33s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/2/artifact/out/Dockerfile
GITHUB PR #4374
Optional Tests dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname Linux 359669dceff6 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / abf9bdc
Default Java Red Hat, Inc.-1.8.0_332-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/2/testReport/
Max. process+thread count 519 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/2/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@GauthamBanasandra GauthamBanasandra changed the title HDFS-16604. Upgrade GoogleTest to 1.11.0 HDFS-16604. Install gtest via FetchContent_Declare in CMake May 30, 2022
@GauthamBanasandra GauthamBanasandra marked this pull request as ready for review May 30, 2022 15:08
@GauthamBanasandra
Copy link
Member Author

GauthamBanasandra commented May 30, 2022

Just to explain the failures in the CI run above - #4374 (comment),

image
The trunk has failed to build hadoop-hdfs-native-client. This failure is expected since CMake is unable to checkout release-1.10.0 in gtest causing the build to fail altogether. (Which is what has been fixed by this PR).

image
These warnings aren't introduced by this PR. They exist in the trunk as well. These warnings aren't showing up in the trunk because hadoop-hdfs-native-client isn't getting built on trunk in the first place.

In conclusion, once we merge this PR to trunk, the above failures won't show up for those PRs which are newly created, or those which gets rebased on top of trunk.

@hadoop-yetus

This comment was marked as outdated.

@hadoop-yetus

This comment was marked as outdated.

@hadoop-yetus

This comment was marked as outdated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 39m 26s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 37m 52s trunk passed
-1 ❌ compile 0m 47s /branch-compile-hadoop-hdfs-project_hadoop-hdfs-native-client.txt hadoop-hdfs-native-client in trunk failed.
+1 💚 mvnsite 0m 47s trunk passed
+1 💚 javadoc 0m 50s trunk passed
+1 💚 shadedclient 59m 1s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 24s the patch passed
+1 💚 compile 3m 40s the patch passed
-1 ❌ cc 3m 40s /results-compile-cc-hadoop-hdfs-project_hadoop-hdfs-native-client.txt hadoop-hdfs-project_hadoop-hdfs-native-client generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 💚 golang 3m 40s the patch passed
+1 💚 javac 3m 40s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 27s the patch passed
+1 💚 javadoc 0m 22s the patch passed
+1 💚 shadedclient 19m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 34m 53s hadoop-hdfs-native-client in the patch passed.
+1 💚 asflicense 0m 50s The patch does not generate ASF License warnings.
161m 16s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/6/artifact/out/Dockerfile
GITHUB PR #4374
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets cc golang
uname Linux d82b0a18e275 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 00f704b
Default Java Red Hat, Inc.-1.8.0_332-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/6/testReport/
Max. process+thread count 545 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4374/6/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@aajisaka aajisaka merged commit 4496e2d into apache:trunk Jun 1, 2022
@aajisaka
Copy link
Member

aajisaka commented Jun 1, 2022

Merged. Thank you @GauthamBanasandra @goiri

@GauthamBanasandra GauthamBanasandra deleted the fix-gtest branch June 1, 2022 05:06
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
)

Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants