[gettext] msgfmt (and other gettext bin tools) missing on macOS & Windows

[wrobelda]

Describe the bug
macOS nor Windows come with gettext by default. There is a gettext port in vcpkg, but it doesn't ship the \bin tools. The vcpkg_find_acquire_program() does not offer it, either. Because of that reason, ports that require msgfmt binary tools (e.g. kf5i18n) fail to build on macOS and Windows.

What's even more puzzling, though, is that the vcpkg OS X Azure Pipeline builds it just fine, despite that gettext is not one of the home-brew prerequisites installed by vagrant. EDIT: these pipeline builds were in fact false-positives and were never actually executed.

Environment

• OS: macOS 10.15.6, as well as Windows 10
• Compiler: Apple clang version 11.0.3 (clang-1103.0.32.62)

To Reproduce
Steps to reproduce the behavior:

1. Make sure you do not have a gettext brew package installed
2. vcpkg install kf5i18
3. See the error.

Expected behavior

1. Azure Pipeline build should fail just like the local build does
2. vcpkg_find_acquire_program() should include gettext tools

Alternatively, the way that OS X Azure Pipeline delivers a functioning set of gettext tools should be explicitly documented. as explained above, this was a false-positive.

Failure logs

CMake Error at /Volumes/External/Sourcecode/vcpkg/downloads/tools/cmake-3.17.2-osx/cmake-3.17.2-Darwin-x86_64/CMake.app/Contents/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
  Could NOT find Gettext (missing: GETTEXT_MSGMERGE_EXECUTABLE
  GETTEXT_MSGFMT_EXECUTABLE)
Call Stack (most recent call first):
  /Volumes/External/Sourcecode/vcpkg/downloads/tools/cmake-3.17.2-osx/cmake-3.17.2-Darwin-x86_64/CMake.app/Contents/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
  /Volumes/External/Sourcecode/vcpkg/downloads/tools/cmake-3.17.2-osx/cmake-3.17.2-Darwin-x86_64/CMake.app/Contents/share/cmake-3.17/Modules/FindGettext.cmake:81 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  /Volumes/External/Sourcecode/vcpkg/scripts/buildsystems/vcpkg.cmake:478 (_find_package)
  /Volumes/External/Sourcecode/vcpkg/buildtrees/kf5i18n/x64-osx-dbg/cmake/KF5I18nMacros.cmake:5 (find_package)
  CMakeLists.txt:66 (include)

[wrobelda]

I just realized that the "CASCADE" status effectively means "SKIPPED because of missing dependency". IMHO a build should fail in such case, since the ports I modified were not built at all.

Anyhow, the issue still stands – these tools are missing on macOS.

[cenit]

agree with CASCADE status. It already misled me few times...

[wrobelda]

Please note we have the same issue on Windows; I updated the title accordingly.

[JackBoosY]

We may need to fix windows build instead of add it to vcpkg_find_acquire_program.

[wrobelda]

We may need to fix windows build instead of add it to vcpkg_find_acquire_program.

I tried building it on macOS and it failed, too. My ports depend on it so I added it for the time being, but I agree it would be best to include those execs in the gettext itself.

[wrobelda]

@strega-nil can you please re-open this? Unfortunately this issue wasn't fixed by #11776. Please see #13532 for ongoing discussion.

[PhoebeHui]

The feature 'tools' has been added to gettext which contains autopiont and msgmerge via PR #17010, however, I don't know how to consume it in related to ports yet.

Related to #17227 and #17278

[dg0yt]

• To get the tools added in [gettext] Add feature tools #17010, users need to build, or ports need to host-depend on, gettext[tools].
• [vcpkg] Add missing autoconf dependencies to osx provisioning #17227/Add missing autoconf dependencies to Linux provisioning #17278 is only to help CI reliably build packages which run autoconf and use autopoint during that step. It doesn't help vcpkgs users who wish to build such packages or who want to use the tools. (However, users might install the tools by system package managers in the same way as this PR does.)
• Add missing autoconf dependencies to Linux provisioning #17278[gettext] Add "autopoint" tool to gettext[core] #17317 is an alternative way to generally help with the autopoint step. It doesn't provide any of the other tools, but is cheap to use as host-depends when needed. So it help vcpkgs users who wish to build such packages, but it doesn't help those who want to use the tools.

[dg0yt]

The feature 'tools' has been added to gettext which contains autopiont and msgmerge via PR #17010, however, I don't know how to consume it in related to ports yet.

I think what kf5i18n needs is a host-dependency on gettext[tools]. (What it has now is a target dependency on the default feature.)

[JackBoosY]

@dg0yt No, it doesn't work, I already checked that.

[dg0yt]

@JackBoosY Perhaps FindGettext needs some help to locate the tools in vcpkg's installation? At least for cross-builds, it might probably fail anyway, because it uses find_program to look for the tools, and this will look for target programs IIUC.
Why not let the portfile run find_program, and pass the result as GETTEXT_MSGMERGE_EXECUTABLE to cmake when configuring the source project?

[JackBoosY]

@dg0yt Because not only our port, but also the custom project will call the configuration file of kf5i18n, so I think we should do some magic on its cmake configuration file.

[dg0yt]

IIUC the gettext port's cmake wrapper is installed for intl, but kf5i18n needs a wrapper for gettext to be provided by gettext[tools].

[wrobelda]

This has been stale for a while and still a blocker for a whole bunch of the libraries I have lined up for inclusion into vcpkg. I understand that the latest is that that gettext[tools] would need to provide a wrapper for gettext, so that using kf5i18n (or any other port that uses gettext's formatting tools) as a dependency works, or I have to patch the kf5i18n's CMake configs directly? I can get away with the latter, but obviously the former solution would be cleaner and universal.

[dg0yt]

WRT to the headline, the tools are no longer missing. (They even build on Linux if you want.) But...

• The tools are in tools/gettext/bin.
• FindGettext.cmake uses plain find_program.
• tools/gettext/bin is neither in ENV{PATH} nor in CMAKE_PROGRAM_PATH.
• So FindGettext.cmake is bound to fail.

Basically, this is a recurring issue, not just affecting gettext[tools]. I suppose at least CMAKE_PROGRAM_PATH should be generally adjusted to include bin and debug suffixes if they exist.
Now we cannot know the proper fix unless #13649 (@wrobelda) or #17607 (@dg0yt) are answered.

[wrobelda]

tools/gettext/bin is neither in ENV{PATH} nor in CMAKE_PROGRAM_PATH.

OK, so as you noticed yourself, this is actually same recurring issue I commented on before and you already linked to. Good job at summarizing everything in #17607. Let's hope that this is picked up by the core team eventually.

[dg0yt]

#18159 was merged, so maybe this resolved?

[JackBoosY]

Ping @wrobelda for response.

[JackBoosY]

Okay, please reopen this issue if it's not resolved.

[ephphatha]

vcpkg install gettext[tools]:triplet isn't enough on its own to make find_package(Gettext) succeed for windows/msvc builds still, looks like a solution to #17607 is needed too.

If I leave the default installation and integration active gettext tools are built and placed in vcpkg\installed\<triplet>\tools\gettext\bin\, however only vcpkg\installed\<triplet>\tools\ is searched when looking for msgfmt and msgmerge. Copying the relevant executables from gettext\bin\ into tools "works" but it's ugly and takes a bit to get everything that's needed.

Edit: Adding PATH_SUFFIXES bin to the two find_program calls at the start of FindGettext.cmake is enough to get a cmake project in VS2022 to succeed with find_package(Gettext) but I'm not sure if that's the appropriate change to make.

[cenit]

without taking away any importance to #17607, maybe we can write specifically for this case a wrapper that finds the executables in the positions that we already know and then launches the real find_package. In this way, find_program calls will already find cache variables set up and work, without any requirements on upstream cmake modules

[dg0yt]

While we don't have a vcpkg wrapper (which would need to be in the target system), we do have a vcpkg config file (which is applied when using a host dependency on gettext[tools]).

My PR #18159 which added the config file was initially submitted with a gettext wrapper installed in the target system. But this concept didn't convicnce the reviewers: installing a wrapper for the target triplet which relies on tools installed for the host triplet.
92149d4