Cabal >= 3.14 doesn't add build-tools with an absolute Path for Setup.hs test and v1-test

[sternenseemann]

Describe the bug
cabal-install/Cabal add build-tool-depends to PATH in the environment of the running test suite since Cabal 3.14. For v2-test, this uses absolute paths while v1-test and Setup.hs use relative paths.

This breaks test suites that rely on this feature to find a certain executable and change directory during the course of their execution (e.g. use temporary directories). (See below for a list of affected Hackage packages.)

To Reproduce

1. Clone https://github.com/sternenseemann/reproducer-cabal-build-tools-test-suite.
   This is a simple executable that prints "hello world" and a test suite that uses it as build tool. It looks up its location and then runs it twice: once from the initial working directory and once from ./test.
   The latter will fail if the path to the build tool is not absolute.
2. Run cabal v2-test --test-show-details=streaming and observe everything working as expected:

[("PATH","/home/lukas/src/hs/reproducer-cabal-build-tools-test-suite/./dist-newstyle/build/x86_64-linux/ghc-9.12.3/reproducer-cabal-build-tools-test-suite-0/t/reproducer-cabal-build-tools-test-suite-test/build/build-tool:/home/lukas/src/hs/reproducer-cabal-build-tools-test-suite/dist-newstyle/build/x86_64-linux/ghc-9.12.3/reproducer-cabal-build-tools-test-suite-0/x/build-tool/build/build-tool:/nix/store/92jwzc69lwmlzr80vv3ssd50i0b19cbk-groff-1.23.0/bin:/nix/store/x12lw455sq6qy2wcya85d7rb88ybc3df-bash-interactive-5.3p9/bin:/nix/store/c2p7haf4zzkbrir9zs662r68c5dmylbq-patchelf-0.15.2/bin:/nix/store/a245z3cvf9x9sn0xlk6k8j9xhxbhda1z-gcc-wrapper-15.2.0/bin:/nix/store/mjf8jlq9grydcdvyw6hb063x5c34g5gf-gcc-15.2.0/bin:/nix/store/0bdqq2z98kg2hfn3k60if6pb5fd5p10h-glibc-2.42-47-bin/bin:/nix/store/i2vmgx46q9hd3z6rigaiman3wl3i2gc4-coreutils-9.9/bin:/nix/store/i6ppbrlpp6yki8qvka7nyv091xa8dchx-binutils-wrapper-2.44/bin:/nix/store/47mn80zqpygykqailwzw8zlag4cgl75q-binutils-2.44/bin:/nix/store/q6lnm5x1ji2m8vhsb43xf48vr6yd2ysd-ghc-9.12.3/bin:/nix/store/z2903hglhpxv4hyczli0ncdf09wmma92-ncurses-6.6/bin:/nix/store/h0x0p1ikp9svk1ri7xxacaczqawyz4sm-numactl-2.0.18/bin:/nix/store/i2vmgx46q9hd3z6rigaiman3wl3i2gc4-coreutils-9.9/bin:/nix/store/16wfacfgap3chf7mcjnd8dwi85dj4qqi-findutils-4.10.0/bin:/nix/store/3p87h6dn5i87i3iq9364imzbqgwvkg2p-diffutils-3.12/bin:/nix/store/ryz8kcrm2bxpccllfqlb7qldsfnqp5c2-gnused-4.9/bin:/nix/store/02vv0r262agf9j5n2y1gmbjvdf12zkl0-gnugrep-3.12/bin:/nix/store/2xq9rayckw8zq26k274xxlikn77jn60j-gawk-5.3.2/bin:/nix/store/qyg62bc2xnpwz0fa9prqxvvk00zj4g9q-gnutar-1.35/bin:/nix/store/84yyzmxs7mb8nhkvcfv9n1l9irpb6mnq-gzip-1.14/bin:/nix/store/90yw24gqmwph4xjp4mqhpx1y1gcrvqla-bzip2-1.0.8-bin/bin:/nix/store/vbah5c4rzy1q1hbqhginyxjhj8d4dj8j-gnumake-4.4.1/bin:/nix/store/f15k3dpilmiyv6zgpib289rnjykgr1r4-bash-5.3p9/bin:/nix/store/wwij6563c6wbg4kzgjhng7vlhf7api19-patch-2.8/bin:/nix/store/zys6d102zp171wpwcs08g632886w2qxs-xz-5.8.2-bin/bin:/nix/store/nyy0bvgjwd98x7ih8pl6pr79qjljgsf7-file-5.45/bin:/run/wrappers/bin:/home/lukas/.nix-profile/bin:/nix/profile/bin:/home/lukas/.local/state/nix/profile/bin:/etc/profiles/per-user/lukas/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin")]
/home/lukas/src/hs/reproducer-cabal-build-tools-test-suite/dist-newstyle/build/x86_64-linux/ghc-9.12.3/reproducer-cabal-build-tools-test-suite-0/x/build-tool/build/build-tool/build-tool is absolute
Hello World!
Hello World!

4. Run cabal v1-test --show-details=streaming (or the equivalent Setup.hs commands) and observe:

[("PATH","dist/build/build-tool:/nix/store/92jwzc69lwmlzr80vv3ssd50i0b19cbk-groff-1.23.0/bin:/nix/store/x12lw455sq6qy2wcya85d7rb88ybc3df-bash-interactive-5.3p9/bin:/nix/store/c2p7haf4zzkbrir9zs662r68c5dmylbq-patchelf-0.15.2/bin:/nix/store/a245z3cvf9x9sn0xlk6k8j9xhxbhda1z-gcc-wrapper-15.2.0/bin:/nix/store/mjf8jlq9grydcdvyw6hb063x5c34g5gf-gcc-15.2.0/bin:/nix/store/0bdqq2z98kg2hfn3k60if6pb5fd5p10h-glibc-2.42-47-bin/bin:/nix/store/i2vmgx46q9hd3z6rigaiman3wl3i2gc4-coreutils-9.9/bin:/nix/store/i6ppbrlpp6yki8qvka7nyv091xa8dchx-binutils-wrapper-2.44/bin:/nix/store/47mn80zqpygykqailwzw8zlag4cgl75q-binutils-2.44/bin:/nix/store/q6lnm5x1ji2m8vhsb43xf48vr6yd2ysd-ghc-9.12.3/bin:/nix/store/z2903hglhpxv4hyczli0ncdf09wmma92-ncurses-6.6/bin:/nix/store/h0x0p1ikp9svk1ri7xxacaczqawyz4sm-numactl-2.0.18/bin:/nix/store/i2vmgx46q9hd3z6rigaiman3wl3i2gc4-coreutils-9.9/bin:/nix/store/16wfacfgap3chf7mcjnd8dwi85dj4qqi-findutils-4.10.0/bin:/nix/store/3p87h6dn5i87i3iq9364imzbqgwvkg2p-diffutils-3.12/bin:/nix/store/ryz8kcrm2bxpccllfqlb7qldsfnqp5c2-gnused-4.9/bin:/nix/store/02vv0r262agf9j5n2y1gmbjvdf12zkl0-gnugrep-3.12/bin:/nix/store/2xq9rayckw8zq26k274xxlikn77jn60j-gawk-5.3.2/bin:/nix/store/qyg62bc2xnpwz0fa9prqxvvk00zj4g9q-gnutar-1.35/bin:/nix/store/84yyzmxs7mb8nhkvcfv9n1l9irpb6mnq-gzip-1.14/bin:/nix/store/90yw24gqmwph4xjp4mqhpx1y1gcrvqla-bzip2-1.0.8-bin/bin:/nix/store/vbah5c4rzy1q1hbqhginyxjhj8d4dj8j-gnumake-4.4.1/bin:/nix/store/f15k3dpilmiyv6zgpib289rnjykgr1r4-bash-5.3p9/bin:/nix/store/wwij6563c6wbg4kzgjhng7vlhf7api19-patch-2.8/bin:/nix/store/zys6d102zp171wpwcs08g632886w2qxs-xz-5.8.2-bin/bin:/nix/store/nyy0bvgjwd98x7ih8pl6pr79qjljgsf7-file-5.45/bin:/run/wrappers/bin:/home/lukas/.nix-profile/bin:/nix/profile/bin:/home/lukas/.local/state/nix/profile/bin:/etc/profiles/per-user/lukas/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin")]
dist/build/build-tool/build-tool is not absolute
reproducer-cabal-build-tools-test-suite-test: Uncaught exception ghc-internal:GHC.Internal.IO.Exception.IOException:

dist/build/build-tool/build-tool: createProcess: posix_spawnp: does not exist (No such file or directory)

While handling createProcess: posix_spawnp: does not exist (No such file or directory)

HasCallStack backtrace:
  ioError, called at libraries/process/System/Process/Common.hs:240:16 in process-1.6.26.1-27cf:System.Process.Common

Hello World!

Note that with GHC 9.12.2/Cabal 3.14.1.0 this issue is not quite always reproducible. Before #10827, the duplicate PATH entries in the environment could cause the test suite to never see the modified PATH environment variable. This has masked the issue in Nixpkgs so far because we add the binary to PATH manually as well (hence my comment).

cc @mpickering

Expected behavior

I see two options:

1. Don't make build-tool-depends available at test suite run time where the working directory may change arbitrarily.
2. Add an absolute path to PATH like for v2-test.

System information

• NixOS
• Cabal 3.14.1.0, 3.14.2.0, GHC 9.12.2, 9.12.3

Additional context

The problem is that there are at least two projects in the wild that rely on PATH being updated with build tools in order to be able to test the package's main executable:

• cabal-add
• fourmolu

This is unfortunate since it's not really what the feature is intended for, but there's also hardly any better option with what Cabal offers at the moment.