w64devkit and GNU Autotools #50
Comments
|
I think it would be great if autotools could work using busybox-w32 shell and w64devkit tools.
What actually doesn't work? I presume semicolon path separator is not great, and/or possibly colon at absolute paths, and/or absolute paths don't necessarily begin with slash? (and something with uname?) It was mentioned before, but allowing some prefix as virtual root and make all the paths normal posix paths can help a lot in many things which don't expect windows path [separator], in addition to taking colon path separator. Something like The git-for-windows busybox-w32 downstream fork already has such changes (check the |
|
Interesting, what did it take to get the Expat build working? In the past
I've only observed configure scripts immediately fail and gave up on the
idea. I tried it just now with Expat using FRP-4882-g6e0a6b7e5, and after
supplying --build it does indeed seem, as avih mentioned, semicolon path
separation is the first issue, based on the output from running the
configure script with "sh -x". I guess you either faked a colon-delimited
PATH for the configure script or hacked something into busybox-w32?
It would be nice, especially if it opens the doors to bootstrapping more
of w64devkit on itself, which is currently limited to busybox-w32, Vim,
Universal Ctags, Cppcheck, PDCurses, GNU Make, and u-config. It's a lot
easier to debug issues in tools I can build natively, so I don't need to
go back and forth cross-compiling between tests. (This has been a serious
impediment debugging GDB itself, for instance.) Otherwise it's not a big
deal for me personally.
Is there anything we could do the other way, to make the environment more
tenable for Autoconf scripts? It would be convenient if existing configure
scripts worked rather than wait for Autoconf support to percolate, which
would take years. The Autoconf on my laptop is over a decade old (2.69),
and I'm running the latest Debian, so that's potentially a very long time
to wait. This adaptation could perhaps take the form of an ash Autoconf
compatibility mode, or a shell wrapper through which configure scripts are
invoked (including recursive invocations).
$ sh --autoconf ./configure --prefix=...
$ acrun ./configure --prefix=...
I imagine these would, at the very least, provide a colon-delimited PATH
for just the script itself, but not for any programs it runs besides shell
built-ins. Maybe it would even need to virtualize the filesystem into
something more unixy just for the script? Though at that point maybe it's
just time to use MSYS2/Cygwin instead.
|
|
The details are in the busybox-w32 issue mentioned above. The tl;dr is that it's a stock w64devkit but you need to source this script before running export PATH_SEPARATOR=';'
export ac_executable_extensions='.exe'
export build_alias="$(uname -m)-pc-mingw64"
if [ -f configure ]
then
echo "Converting configure..."
sed -i 's/func_convert_file_msys_to_w32/func_convert_file_noop/' configure
fi |
|
Doh! I went diving in and forgot you had provided a link.
It seems perfectly reasonable to set these three variables as indicated in
w64devkit.exe if not already set. The first one seems like a good idea
even without considering Autoconf. I found I could skip the sed step by
also exporting lt_cv_to_{host,tool}_file_cmd to that noop. If that works
generally then all can be accomplished using environment variables.
|
|
I just tried Binutils, and it has func_to_host_path{list,} that hardcodes
two more "cmd //c" but this time behind a *mingw* case at "run time". If I
fix them manually (across each libtool copy) then it builds successfully.
I wonder if there's a more general solution: a cmd.exe in w64devkit that
filters any "//c" before invoking the real cmd.exe. Or alternatively do it
in ash.
|
|
I whipped one up to try it out: cmd.c. With that |
// Converts libtool's Cygwin-style "cmd //c ..." to "cmd /c ..."
// $ cc -nostartfiles -o cmd.exe cmd.cThis is not a cygwin thing, it's an MSYS2 thing. In cygwin arguments are unmodified, and if you want to pass a Windows path to a Windows utility then you should convert the path yourself, using $ /cygdrive/c/Windows/notepad.exe "$(cygpath -w /cygdrive/c/foo.txt)"In MSYS2. however, they want to make it easier to integrate with Windows programs, so they do (at least these) three hacks:
This behavior is baked into MSYS2's This can be tested with the following #include <stdio.h>
int main(int argc, char **argv) {
while (*argv)
printf("[%s]\n", *argv++);
return 0;
}and $ ./args.exe foo /c /foo //bar foo/bar
[D:\run\utils\args.exe]
[foo]
[C:/]
[T:/msys64/foo]
[/bar]
[foo/bar]However, without a windows program (or with a native MSYS2 binary) the args are unmodified: $ printf "[%s]\n" foo /c /foo //bar foo/bar
[foo]
[/c]
[/foo]
[//bar]
[foo/bar]All this is very unfortunate together with the (traditional) windows switches which use I'm guessing they manage to upstream this behavior into autotools (or only their |
Indeed. For comparison, busybox-w32 @skeeto Excellent progress on those builds! This looks very promising. |
|
I do wonder though why they use Is there some functionality which Presumably, if they apply the So why
It's a shame that MSYS2 upstreamed "MINGW" prefix and associated it with hacked paramaters mangling. It really should have been something like "MSYS2-MINGW...", because not all MINGW setups (current or future) would necessarily hack the arguments like MSYS2 does... |
I was puzzled by that too. The only cases I've seen where this is used are of the form: A subshell to run This message from the author of the patch which added the It seems |
Well, MSYS2 does have cygpath, and converting
However, if they did that, which would be the correct way to handle path conversion instead of relying on some magic MSYS2 arguments hacks, then it still won't work in busybox-w32 (unless So yeah, being able to detect the usage pattern and possibly replace it automatically with something which works in busybox-w32 should work. Alternatively, maybe they do have some simple mode which doesn't rely on MSYS2 hacks, which could be enabled/enforced using some env vars, and which would happen to work in busybox-w32 (considering it's paths are indeed non-posix...). |
Cygwin doesn't depend on MSYS2 hacks. Using |
Hmm.. that's a bit hacky I think, and it's possible IMHO that some things could break either during configure or the actual build later, even if they didn't for expat. However, it seems that, at least for expat, the env vars and the cmd //c thing are enough, so a script like this, saved at the PATH as e.g. The script: #!/bin/sh
# run autotools "configure" script in w64devkit env, using busybox-w32 sh,
# utilizing the MSYS2 MINGW code paths, and work around the following:
# - export some env vars which bypass the detection by "configure".
# - "cmd //c ..." is replaced with "cmd /c ...". originally //c works around
# MSYS2 auto args copnversion to get "/c" - not needed with busybox-w32.
# The modification is written to a temporary file at the same dir, which is
# deleted [before startup and] after it's invoked.
echo() { printf %s\\n "$*"; }
error() {
[ "${1+x}" ] && >&2 echo "${0##*/}: $*"
>&2 echo "Usage: ${0##*/} path/to/autotools-configure [ARG...]"
exit 1
}
[ "${1-}" ] || error
[ -e "$1" ] || error cannot find file -- "$1"
conf=$1
bbconf=$conf.bb.tmp
rm -f -- "$bbconf" || error "cannot remove file -- $bbconf"
shift
# The following might be needed too in some cases, but currently not applied
# 's/func_convert_file_msys_to_w32/func_convert_file_noop/'
sed -e 's/cmd \/\/c/cmd \/c/g' < "$conf" > "$bbconf" \
|| error "cannot create file -- $bbconf"
chmod +x "$bbconf"
export PATH_SEPARATOR=';'
export ac_executable_extensions='.exe'
export build_alias="$(uname -m)-pc-mingw64"
"$bbconf" "$@"
e=$?
rm -- "$bbconf"
exit "$e"This makes it explicit that some conversion is applied. However, if indeed the //c and exports are enough, then it would be more user friendly to export the stuff unconditionally automatically, and handle the //c thing using some script/binary wrapper. |
|
Unfortunately fixing the root configure script is often insufficient,
which is what lead to my cmd.exe wrapper idea. Namely that approach
doesn't handle:
* recursive configure, common in GNU, including Bintutils, GCC, and GDB
* instances of "cmd //c" outside configure scripts
Binutils, for instance, has both. Its root configure script recursively
runs 14 other configure scripts. These each generate a "libtool" script
via "ltmain.sh" which has the actual hard-coded instance of "cmd //c". A
fix-up script would need to be able to figure this out.
|
Right. I think it can also be implemented as a shell script, either named If the cmd //c thing is the main/only issue, then a wrapper executable (sh/binary) and the exported env should have it covered. |
|
Here's a patch to busybox-w32 diff --git a/shell/ash.c b/shell/ash.c
index 742067216..886498640 100644
--- a/shell/ash.c
+++ b/shell/ash.c
@@ -8970,6 +8970,21 @@ static int builtinloc = -1; /* index in path of %builtin, or -1 */
static void
tryexec(IF_FEATURE_SH_STANDALONE(int applet_no,) const char *cmd, char **argv, char **envp)
{
+#if ENABLE_PLATFORM_MINGW32
+ int a;
+
+ if (strcmp(argv[0], "cmd") == 0 &&
+ argv[1] && strcmp(argv[1], "//c") == 0 &&
+ argv[2] && strcmp(argv[2], "echo") == 0 &&
+ argv[3] && !argv[4] && (a = find_applet_by_name("echo")) > 0) {
+ argv += 2;
+ cmd = "echo";
+#if ENABLE_FEATURE_SH_STANDALONE
+ applet_no = a;
+#endif
+ }
+#endif
+
#if ENABLE_FEATURE_SH_STANDALONE
if (applet_no >= 0) {
# if ENABLE_PLATFORM_MINGW32 |
I didn't try that, but (together with the exported env vars) I did try a So I tried the same shell script, this time saved as #!/bin/sh
# save me as "cmd.exe" and place it early in PATH
# autotools assumes MINGW setup uses MSYS[2] env, and uses "cmd //c echo..."
# to invoke "cmd /c echo..." (MSYS converts //c into /c for windows prog args).
# this breaks in busybox-w32 sh, so replace the //c with normal /c
# only handle exactly cmd //c echo...
# could be enhance for more cases, but for autotools this seems enough
# and without breaking "cmd.exe" arguments in general
case ${COMSPEC-} in *\\cmd.exe)
if [ "${1-}" = //c ] && [ "${2-}" = echo ]; then
shift
set -- /c "$@"
fi
esac
exec "$COMSPEC" "$@"And this does seem to work. I'm guessing this could break with unicode paths, but then again, if unicode paths are used then I'm guessing it would break elsewhere too regardless of this EDIT: |
Introduce a "system-wide" profile loaded before ~/.profile, and populate it with variables to guide GNU Autoconf. In general, "configure" scripts should now work out-of-the-box. Libtool assumes the host environment is MSYS2 and will be confused by Windows-style command switches. The "/c" in "cmd /c" looks like a path, which MSYS2 incorrectly decodes to "c:/". Anticipating this, libtool encodes these calls as "cmd //c" which does not work outside MSYS2. A busybox-w32 patch makes it behave like MSYS2 in just this one case.
Libtool assumes the host environment is MSYS2 and will be confused by Windows-style command switches. The "/c" in "cmd /c" looks like a path, which MSYS2 incorrectly decodes to "c:/". Anticipating this, libtool encodes these calls as "cmd //c" which does not work outside MSYS2. A busybox-w32 patch makes it behave like MSYS2 in just this one case. Adds 88-96 bytes. (GitHub issue #297 and skeeto/w64devkit#50)
Over at rmyorston/busybox-w32#297 we've been working on getting stuff that uses a
configurescript to build with w64devkit.With a bit of hackery I've managed to get Expat to build. (Chosen as a guinea pig because it's included with w64devkit and it's quite small.)
GNU Autotools support for Windows platforms is mostly restricted to Cygwin/MSYS2. This sort of works for w64devkit (hence the hackery) but what's really needed is proper support for the w64devkit build environment.
Is there any interest in prodding the GNU Autotools people to do this?
The text was updated successfully, but these errors were encountered: