Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overlay triplets test #961

Merged
merged 4 commits into from
Nov 8, 2022
Merged

Conversation

pkgw
Copy link
Collaborator

@pkgw pkgw commented Oct 27, 2022

Try using my customized vcpkg with --overlay-triplets and a Windows/static/release build to see if our builds speed up!

@pkgw pkgw force-pushed the overlay-triplets-test branch 2 times, most recently from b706964 to 5704e75 Compare October 27, 2022 02:13
@pkgw pkgw closed this Oct 27, 2022
@pkgw pkgw reopened this Oct 27, 2022
@codecov
Copy link

codecov bot commented Oct 27, 2022

Codecov Report

❗ No coverage uploaded for pull request base (master@204f033). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head af495c3 differs from pull request most recent head caab2c2. Consider uploading reports for the commit caab2c2 to get more accurate results

@@            Coverage Diff            @@
##             master     #961   +/-   ##
=========================================
  Coverage          ?   45.18%           
=========================================
  Files             ?      148           
  Lines             ?    62138           
  Branches          ?        0           
=========================================
  Hits              ?    28077           
  Misses            ?    34061           
  Partials          ?        0           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pkgw
Copy link
Collaborator Author

pkgw commented Oct 27, 2022

Initial findings are quite positive: the Windows build of vcpkg deps speeds up considerably. In my test run here the "Build vcpkg deps" step took 27 minutes, while in recent runs it has taken ~40-55 minutes.

However, the Windows vcpkg build of Tectonic itself failed because of a problem with graphite2:

thread 'main' panicked at 
'failed to load package graphite2 from vcpkg: 
  Could not find library in Vcpkg tree package 
  graphite2 is not installed for vcpkg triplet x64-windows-static', 
  D:\a\1\s\crates\dep_support\src\lib.rs:130:37

Hopefully this is something that can be deal with easily. The logs from building graphite2 here are:

Installing 19/22 graphite2:x64-windows-static-release...
Building graphite2[core]:x64-windows-static-release...
-- [OVERLAY] Loading triplet configuration from: \\?\D:\a\1\s\dist\vcpkg-triplets\x64-windows-static-release.cmake
-- Downloading https://github.com/silnrsi/graphite/archive/92f59dcc52f73ce747f1cdc831579ed2546884aa.tar.gz -> silnrsi-graphite-92f59dcc52f73ce747f1cdc831579ed2546884aa.tar.gz...
-- Extracting source D:/a/1/s/target/vcpkg/downloads/silnrsi-graphite-92f59dcc52f73ce747f1cdc831579ed2546884aa.tar.gz
-- Applying patch disable-tests.patch
-- Using source at D:/a/1/s/target/vcpkg/buildtrees/graphite2/src/d2546884aa-145bc79c0d.clean
-- Found external ninja('1.10.2').
-- Configuring x64-windows-static-release
-- Building x64-windows-static-release-rel
-- Installing: D:/a/1/s/target/vcpkg/packages/graphite2_x64-windows-static-release/share/graphite2/copyright
-- Fixing pkgconfig file: D:/a/1/s/target/vcpkg/packages/graphite2_x64-windows-static-release/lib/pkgconfig/graphite2.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-9.0.0.6373.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at D:/a/1/s/target/vcpkg/downloads/tools/msys2/9a1ec3f33446b195
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: "C:\Users\VssAdministrator\AppData\Local\vcpkg\archives\50\5071790e1ce98663ca69822b9ac76e199606bf1b5c3c299cf4f7734da9ab8ef8.zip"
Elapsed time to handle graphite2:x64-windows-static-release: 10.81 s

Nothing obviously messed up here; it's the same as the standard build, minus debug variants!

Also note that gperf and icu still seem to build debug variants.

This is basically a tactic to speed up the Windows builds. By default, vcpkg
builds both debug and release versions of requested libraries. There's probably
some good reason for that, but for us it just makes the builds a lot slower than
they need to be. We can avoid that by providing a custom "triplet" file that
disables the debug builds. In order to do this cleanly, we use new support from
cargo-vcpkg that adds support for an "overlay triplets" directory. Builds also
need to set the $VCPKGRS_TRIPLET environment variable so that vcpkg-rs can find
the packages it needs. We add some docs and a diagnostic warning if that may be
what's going on when a vcpkg fails to be found.
@pkgw pkgw force-pushed the overlay-triplets-test branch 2 times, most recently from 415bfde to b3c02e7 Compare November 2, 2022 02:13
@pkgw
Copy link
Collaborator Author

pkgw commented Nov 2, 2022

OK, I think this is working! The Windows vcpkg build completes about twice as quickly as it used to, now. The only thing blocking merger is a new release of cargo-vcpkg that incorporates the overlay-triplets-path functionality.

@pkgw
Copy link
Collaborator Author

pkgw commented Nov 8, 2022

OK, well, no cargo-vcpkg release yet, but I think it will be fine to use the Git version until that happens.

@pkgw pkgw merged commit d86b209 into tectonic-typesetting:master Nov 8, 2022
@pkgw pkgw deleted the overlay-triplets-test branch November 8, 2022 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant