Backport #10731: cabal-install: Be less eager to configure external programs

[mergify]

In configureCompiler the call to configureAllKnownPrograms was too eager. When called it selected the version of tools from PATH (such as alex), and then when a package was configured these tools were passed using --with-alex options to configure, which meant that the build-tool-depends versions were not used. (See #10692)

Why was this call introduced in the first place? Because configureCompiler would a different result depending on whether:

• It is run for the first time, the ProgramDb will contain unconfigured programs.
• It is run subsequently, ProgramDb is read from disk, it does not contain unconfigured programs.

A surgical way to fix this is to avoid configuring the programs, and manually adding back the builtinPrograms to the ProgramDb, so the ProgramDb returned by configureCompiler always contains all the unconfigured programs.

The testcase is not so easy to write because

• The bug only surfaces when the build-tool you are depending on is known (ie alex, happy etc)
• But then it is tricky to write a test, as we can't depend on the known tools or bundle the source for them.
• So we create a fake "alex", which cabal then invokes on a fake ".x" file. This is maybe a bit fragile if the way cabal invokes alex changes in future, but then the test can be modified as well.

Also see #2241 and #9840

Fixes #10692

Please read Github PR Conventions and then fill in one of these two templates.

---

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

---

This is an automatic backport of pull request #10731 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of 1c64bb8 has failed:

On branch mergify/bp/3.14/pr-10731
Your branch is up to date with 'origin/3.14'.

You are currently cherry-picking commit 1c64bb8c1.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   cabal-install/src/Distribution/Client/ProjectPlanning.hs
	new file:   changelog.d/pr-10731

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   cabal-testsuite/PackageTests/ExtraProgPath/setup.out

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[mergify]

This pull request has been removed from the queue for the following reason: checks failed.

The merge conditions cannot be satisfied due to failing checks:

• ❌ docs/readthedocs.org:cabal

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

[Mikolaj]

@mergify dequeue

[mergify]

dequeue

☑️ The pull request is not queued

[Mikolaj]

@mergify rebase

[mergify]

rebase

✅ Branch has been successfully rebased