[vcpkg_acquire_msys,vcpkg-ci-msys2] Ensure msys2 package list freshness #29067
+751
−298
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
MonicaLiu0311
added
the
category:infrastructure
|
Current state: |
|
So libtool in vcpkg was updated after removal from msys2 servers... And now we wait for the next package to break? zlib seems a good candidate. The package downloaded by vcpkg is from 2186 days ago - app. 6 years old. vcpkg has to be some months ahead of package removal from msys2 servers in order to mitigate effects on user which aren't on vcpkg I can try to incrementally improve the situation based on the test ports output.
The test port currently doesn't cover "direct packages" such as pkg-config (which is x86!) from |
It is not really a circular dependency. It is libiconv -> libintl (to get gettext tools) -> libiconv with nls -> libintl with nls. So having just libintl depend on libiconv is ok (since we don't build msys stuff.) Also why do we have ncurses and python from msys ? |
Well, package libintl is really just libintl, depending on libiconv. Tools are in package gettext.
ncurses is a dependency of readline which is a dependeny of gawk which is... (autotools). |
dg0yt
commented
Jan 20, 2023
Comment on lines
-144
to
-145
| URL "https://repo.msys2.org/msys/x86_64/libbz2-1.0.8-3-x86_64.pkg.tar.zst" | ||
| SHA512 955420cabd45a02f431f5b685d8dc8acbd07a8dcdda5fdf8b9de37d3ab02d427bdb0a6d8b67c448e307f21094e405e916fd37a8e9805abd03610f45c02d64b9e |
There was a problem hiding this comment.
libbz2 was listed twice.
Comment on lines
-8
to
-10
| "https://mirror.bit.edu.cn/msys2/" | ||
| "https://mirror.selfnet.de/msys2/" | ||
| "https://mirrors.sjtug.sjtu.edu.cn/msys2/" |
There was a problem hiding this comment.
Removed tier 2 mirrors.
The main switch is now the main URL, mirror.msys2.org.
Comment on lines
162
to
165
| NAME autoconf | ||
| URL "https://mirror.msys2.org/msys/x86_64/autoconf-wrapper-15-1-any.pkg.tar.zst" | ||
| SHA512 7c0f0c619100d05c82409567399253efddee9b39fff7dd772e503770afd4accbae2ce96307a20b076faf2e308413cd25e7405e70969608223bc86c7016ec38b8 | ||
| DEPS autoconf2.71 |
There was a problem hiding this comment.
New autoconf packaging.
Comment on lines
291
to
294
| URL "https://repo.msys2.org/msys/x86_64/pkg-config-0.29.2-4-x86_64.pkg.tar.zst" | ||
| SHA512 9f72c81d8095ca1c341998bc80788f7ce125770ec4252f1eb6445b9cba74db5614caf9a6cc7c0fcc2ac18d4a0f972c49b9f245c3c9c8e588126be6c72a8c1818 | ||
| NAME pkg-config | ||
| URL "https://mirror.msys2.org/msys/x86_64/pkgconf-1.8.0-2-x86_64.pkg.tar.zst" | ||
| SHA512 4fcc1671969098b9b7c79192f90d3c0396ad65cb6efc44fdd7e6e7a37452f0bf4a38fb2ef63e73b0f9883a87c9f248c38b7f66c7d762bfbbd0da1d22aec5b52d |
There was a problem hiding this comment.
pkg-config.exe is now provided from package pkgconf.
There was a problem hiding this comment.
This is a new experimental fast check for PR issues. Please let us know if this bot is helpful!
PRs must add only one version and must not modify any published versions
When making any changes to a library, the version or port-version in vcpkg.json or CONTROL must be modified.
error: checked-in files for libxau have changed but the version was not updated
version: 1.0.9
old SHA: d82d4195d09a023e7a7fbc1c3a726a2f9f917a33
new SHA: 065123320d11c2fc6c6e4f2348695639b4a4b794
Did you remember to update the version or port version?
Use --overwrite-version to bypass this check
***No files were updated***
You have modified or added at least one portfile where deprecated functions are used.
If you feel able to do so, please consider migrating them to the new functions:
vcpkg_install_cmake -> vcpkg_cmake_install (from port vcpkg-cmake)
vcpkg_build_cmake -> vcpkg_cmake_build (from port vcpkg-cmake)
vcpkg_configure_cmake -> vcpkg_cmake_configure (Please remove the option PREFER_NINJA) (from port vcpkg-cmake)
vcpkg_fixup_cmake_targets -> vcpkg_cmake_config_fixup (from port vcpkg-cmake-config)
vcpkg_extract_source_archive_ex -> vcpkg_extract_source_archive
vcpkg_build_msbuild -> vcpkg_install_msbuild
vcpkg_copy_tool_dependencies -> vcpkg_copy_tools
vcpkg_apply_patches should be replaced by the PATCHES arguments to the "extract" helpers (e.g. vcpkg_from_github())
In the ports that use the new function, you have to add the corresponding dependencies:
{
"name": "vcpkg-cmake",
"host": true
},
{
"name": "vcpkg-cmake-config",
"host": true
}The following files are affected:
ports/gettext/portfile.cmake
You have modified or added at least one vcpkg.json where you should check the license field.
If you feel able to do so, please consider adding a "license" field to the following files:
ports/gettext/vcpkg.json
Valid values for the license field can be found in the documentation
|
So this is how libtool looks for the x64-windows import library of ws2_32 and fails:
Relevant libtool variables: # Object dumper program.
OBJDUMP="objdump"
# Method to check whether dependent libraries are shared objects.
deplibs_check_method="file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)"
# Command to use when deplibs_check_method = "file_magic".
file_magic_cmd="\$OBJDUMP -f"
# How to find potential files when deplibs_check_method = "file_magic".
file_magic_glob=""
# Find potential files using nocaseglob when deplibs_check_method = "file_magic".
want_nocaseglob="yes"AFAIU it should use |
7 tasks
|
I manually cancelled the build because it was stuck trying to build gettext for more than a day |
cenit
reviewed
Feb 7, 2023
| @@ -203,7 +203,8 @@ function(vcpkg_configure_make) | |||
| # Pre-processing windows configure requirements | |||
| if (VCPKG_TARGET_IS_WINDOWS) | |||
| if(CMAKE_HOST_WIN32) | |||
| list(APPEND msys_require_packages binutils libtool autoconf automake-wrapper automake1.16 m4 which) | |||
There was a problem hiding this comment.
which was added recently. Why did you remove it?
There was a problem hiding this comment.
It took me a moment to understand which is which...
I was taking the msys2 autotools package as a guideline. And there is ADDITIONAL_MSYS_PACKAGES. And from recent discussion I got the impression that which is also a shell built-in, but IIUC this is wrong at least for bash.
There was a problem hiding this comment.
sorry for the missing code highlighting
Desired changes can be picked in git gui etc..
|
Updated to resolve merge conflicts. |
|
@BillyONeal @ras0219-msft There is only one mirror left (out of eight) providing the outdated/removed grep package, https://mirrors.sjtug.sjtu.edu.cn/msys2/msys/x86_64/grep-3.0-2-x86_64.pkg.tar.xz
In the end this is going to happen because the PR was not merged, for those who don't have an asset cache. |
|
Merging master always means asking for baseline regessions. |
I was going to compare with a recent CI run and merge around that if necessary.
Thank you! |
msvetkin
added a commit
to reMarkable/vcpkg
that referenced
this pull request
Jun 2, 2023
mikedickey
added a commit
to jacktrip/jacktrip
that referenced
this pull request
Jun 2, 2023
Bumping to latest master commit to try pulling in this PR microsoft/vcpkg#29067
seragh
added a commit
to seragh/fcollada
that referenced
this pull request
Jun 3, 2023
microsoft/vcpkg#29067 Signed-off-by: Ralph Sennhauser <ralph.sennhauser@gmail.com>
Closed
msvetkin
added a commit
to reMarkable/vcpkg
that referenced
this pull request
Jun 8, 2023
daschuer
pushed a commit
to daschuer/vcpkg
that referenced
this pull request
Jun 10, 2023
…ss (microsoft#29067) * [vcpkg-ci-msys2] New test port for msys2 freshness * Add features for script updates * Remove duplicate libbz2 entry * Automatically update all msys packages, add autoconf-wrapper * Update primary URL and mirrors * Follow repo renames * Fix dependencies of 'file' * Resolve dependencies at runtime Don't require msys2 packages to be order from dependent to independent. This allows to directly use the packages official dependencies, even if they contain cycles. It also allows to maintain a stable alphabetical order of the package declarations, even if the dependencies change over time. This helps to avoid duplicate declaration. * Fix updating * [gettext] Preset more MSVC checks * Update msys2 packages * gettext fixes * CI * Simplify: Remove update-critical Desired changes can be picked in git gui etc.. * PKGCONFIG moved --------- Co-authored-by: Monica <v-liumonica@microsoft.com> Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
daschuer
pushed a commit
to daschuer/vcpkg
that referenced
this pull request
Jul 9, 2023
…ss (microsoft#29067) * [vcpkg-ci-msys2] New test port for msys2 freshness * Add features for script updates * Remove duplicate libbz2 entry * Automatically update all msys packages, add autoconf-wrapper * Update primary URL and mirrors * Follow repo renames * Fix dependencies of 'file' * Resolve dependencies at runtime Don't require msys2 packages to be order from dependent to independent. This allows to directly use the packages official dependencies, even if they contain cycles. It also allows to maintain a stable alphabetical order of the package declarations, even if the dependencies change over time. This helps to avoid duplicate declaration. * Fix updating * [gettext] Preset more MSVC checks * Update msys2 packages * gettext fixes * CI * Simplify: Remove update-critical Desired changes can be picked in git gui etc.. * PKGCONFIG moved --------- Co-authored-by: Monica <v-liumonica@microsoft.com> Co-authored-by: Billy Robert O'Neal III <bion@microsoft.com>
Lex-2008
pushed a commit
to Lex-2008/vcpkg
that referenced
this pull request
Oct 10, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was waiting for an embarassing incident like the recent removal of the outdated libtool package used by vcpkg from the rolling-release servers of msys2...
Edit: Next incident: msys2-runtime, #30542, #30530, #30546.
Edit: Next incident: python-numpy, #30582. (However, the incident affects tensorflow-cc which is not tested in CI. The issue might not be fixed by this PR.)
Edit: Next incident: grep, #31446, ObjectVision/GeoDMS#225.
Edit: msys2-runtime is still an issue in VS, #31565, #31577, two months after the fix in vcpkg!
This PR is an attempt to help prevent such incidents.
vcpkg_acquire_msys,vcpkg_find_acquire_program(PKGCONFIG)andvcpkg_find_fortran.The check isn't entirely accurate because it would require to know when a package was updated. But this wouldn't matter if updates are done regularly. msys2 packages are removed 1.5 after an update ATM.
update-allfeature.(Note: This even works on linux or osx if installed with
--allow-unsupported.)User may decide to line-wise pick only some of the changes using
git guior similar.vcpkg_acquire_msys.This allows to list the packages in alphabetical order, avoiding multiple definitions with inconsistent dependencies.
msys2. It is not msys.