Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[blaze][ceres][clapack][geogram][lapack][lapack-reference][opencv4][selene] Fix deps issues #13229

Merged
merged 21 commits into from
Sep 5, 2020

Conversation

ras0219
Copy link
Contributor

@ras0219 ras0219 commented Aug 30, 2020

As part of a full rebuild in #13019, several transitive rebuild issues were found.

  1. selene was dynamically detecting the presence of OpenCV, which was not on its dependency list.
  2. The dynamic OpenCV was using find_package() instead of find_dependency()
  3. The dynamic OpenCV was always finding all dependencies instead of just those that were used during build.
  4. CMake's FindHDF5 module requires C language support
  5. ceres was passing REQUIRED into find_dependency (not needed -- find_dependency implies REQUIRED)
  6. FindLAPACK.cmake was sometimes provided by the meta-port lapack, sometimes by the dependency lapack-reference/clapack
  7. The lapack-reference vcpkg-cmake-wrapper was directly including the FindLAPACK, which doesn't respect variables such as QUIET and REQUIRED

Interestingly, the particular issue required a cascade of failures to occur to actually cause a build failure; almost any of these fixes would independently fix the build.

…issues

As part of a full rebuild in microsoft#13019, several transitive rebuild issues were found.

1) selene was dynamically detecting the presence of OpenCV, which was not on its dependency list.
2) The dynamic OpenCV was using `find_package()` instead of `find_dependency()`
3) The dynamic OpenCV was always finding all dependencies instead of just those that were used during build.
4) CMake's FindHDF5 module requires C language support
5) ceres was passing `REQUIRED` into find_dependency (not needed -- find_dependency implies REQUIRED)
6) FindLAPACK.cmake was sometimes provided by the meta-port lapack, sometimes by the dependency lapack-reference/clapack
7) The lapack-reference vcpkg-cmake-wrapper was directly including the FindLAPACK, which doesn't respect variables such as QUIET and REQUIRED

Interestingly, the particular issue required a cascade of failures to occur to actually cause a build failure; almost any of these fixes would independently fix the build.
@cenit
Copy link
Contributor

cenit commented Aug 30, 2020

The same cleaner solution for find_dependencies can be applied to opencv2/3. I used find_package and QUIET to “hack” the system, but it is not as robust as this.
I am jut unsure about the BUILD_opencv_quality switch. Wasn’t contrib feature better? Is opencv quality defined here in the parent script?

@ras0219
Copy link
Contributor Author

ras0219 commented Aug 30, 2020

On uwp, the dependencies are disabled:
https://github.com/microsoft/vcpkg/pull/13229/files#diff-cc80ebd35d7c59af4199c1196aac6692L15-L17

This per-platform logic is already implemented above in the BUILD_opencv_quality switch, so that's why I'm using it: https://github.com/microsoft/vcpkg/pull/13229/files#diff-b2c0c4ecb1a6e09f5fc9b8f84fcdc62bL112-L119

@NancyLi1013 NancyLi1013 added category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist category:port-bug The issue is with a library, which is something the port should already support info:internal This PR or Issue was filed by the vcpkg team. labels Aug 31, 2020
@BillyONeal
Copy link
Member

Hi there @ras0219 :)

zxing-cpp failed on x64-windows-static due to the default build-depends not having protobuf: BillyONeal@f9cc362
It also failed because the bigint dependency failed to be found. A change like the following fixes it: BillyONeal@e80c069 ... but it seems like that shouldn't have been necessary?

@BillyONeal
Copy link
Member

(I have those changes prepared in https://github.com/BillyONeal/vcpkg/commits/robert_review_fix if you want to use it as a base)

@BillyONeal
Copy link
Member

@ras0219 and I talked about this stuff out of band.

@ras0219
Copy link
Contributor Author

ras0219 commented Aug 31, 2020

I'm looking into the dbow2 failure (looks like the transitive dependency through opencv isn't being handled appropriately)

From dbow2:

[7/8] : && /Library/Developer/CommandLineTools/usr/bin/c++ -fPIC -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -dynamiclib -Wl,-headerpad_max_install_names -o libDBoW2.dylib -install_name @rpath/libDBoW2.dylib CMakeFiles/DBoW2.dir/src/BowVector.cpp.o CMakeFiles/DBoW2.dir/src/FBrief.cpp.o CMakeFiles/DBoW2.dir/src/FORB.cpp.o CMakeFiles/DBoW2.dir/src/FeatureVector.cpp.o CMakeFiles/DBoW2.dir/src/QueryResults.cpp.o CMakeFiles/DBoW2.dir/src/ScoringObject.cpp.o /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_calib3dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_cored.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_dnnd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_features2dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_flannd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_highguid.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgcodecsd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgprocd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_mld.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_objdetectd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_photod.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_stitchingd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_videod.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_videoiod.a -llibprotobuf /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgcodecsd.a /Users/vagrant/Data/installed/x64-osx/lib/libjpeg.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdecoderd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdemuxd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libpng16d.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libtiffd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/liblzmad.a /Users/vagrant/Data/installed/x64-osx/lib/libjpeg.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdecoderd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdemuxd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libpng16d.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libtiffd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/liblzmad.a -lm -framework Accelerate -framework AVFoundation -framework CoreGraphics -framework CoreMedia -framework CoreVideo -framework QuartzCore -framework Cocoa /Users/vagrant/Data/installed/x64-osx/debug/lib/libquircd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_calib3dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_features2dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_flannd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgprocd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_cored.a /Users/vagrant/Data/installed/x64-osx/lib/libz.a -framework OpenCL && :
FAILED: libDBoW2.dylib
: && /Library/Developer/CommandLineTools/usr/bin/c++ -fPIC -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -dynamiclib -Wl,-headerpad_max_install_names -o libDBoW2.dylib -install_name @rpath/libDBoW2.dylib CMakeFiles/DBoW2.dir/src/BowVector.cpp.o CMakeFiles/DBoW2.dir/src/FBrief.cpp.o CMakeFiles/DBoW2.dir/src/FORB.cpp.o CMakeFiles/DBoW2.dir/src/FeatureVector.cpp.o CMakeFiles/DBoW2.dir/src/QueryResults.cpp.o CMakeFiles/DBoW2.dir/src/ScoringObject.cpp.o /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_calib3dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_cored.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_dnnd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_features2dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_flannd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_highguid.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgcodecsd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgprocd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_mld.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_objdetectd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_photod.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_stitchingd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_videod.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_videoiod.a -llibprotobuf /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgcodecsd.a /Users/vagrant/Data/installed/x64-osx/lib/libjpeg.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdecoderd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdemuxd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libpng16d.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libtiffd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/liblzmad.a /Users/vagrant/Data/installed/x64-osx/lib/libjpeg.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdecoderd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libwebpdemuxd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libpng16d.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libtiffd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/liblzmad.a -lm -framework Accelerate -framework AVFoundation -framework CoreGraphics -framework CoreMedia -framework CoreVideo -framework QuartzCore -framework Cocoa /Users/vagrant/Data/installed/x64-osx/debug/lib/libquircd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_calib3dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_features2dd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_flannd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_imgprocd.a /Users/vagrant/Data/installed/x64-osx/debug/lib/libopencv_cored.a /Users/vagrant/Data/installed/x64-osx/lib/libz.a -framework OpenCL && :
ld: library not found for -llibprotobuf
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

@ras0219-msft
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

ports/bigint/portfile.cmake Outdated Show resolved Hide resolved
ports/lapack/CONTROL Outdated Show resolved Hide resolved
ports/opencv4/CONTROL Outdated Show resolved Hide resolved
@NancyLi1013 NancyLi1013 added the requires:all-feature-testing vcpkg install port[all features supported by that port] needs to be demonstrated to function label Sep 4, 2020
@BillyONeal BillyONeal changed the title [ceres][lapack][clapack][lapack-reference][opencv4][selene] Fix deps issues [blaze][ceres][clapack][lapack][lapack-reference][opencv4][selene] Fix deps issues Sep 4, 2020
@BillyONeal
Copy link
Member

@BillyONeal BillyONeal changed the title [blaze][ceres][clapack][lapack][lapack-reference][opencv4][selene] Fix deps issues [blaze][ceres][clapack][geogram][lapack][lapack-reference][opencv4][selene] Fix deps issues Sep 4, 2020
Copy link
Contributor

@dan-shaw dan-shaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the library zxing-cpp doesn't have protobuf as a dependency, and opencv installs protobuf. Not sure why it was failing the CI.

@BillyONeal
Copy link
Member

@dan-shaw Good point.

@BillyONeal BillyONeal merged commit bb61fc1 into microsoft:master Sep 5, 2020
@BillyONeal
Copy link
Member

Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:port-bug The issue is with a library, which is something the port should already support category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist info:internal This PR or Issue was filed by the vcpkg team. requires:all-feature-testing vcpkg install port[all features supported by that port] needs to be demonstrated to function
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants