SDK Installer fails due to missing files when building git

[Amorik]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit? 64bit

$ git --version --build-options

git version 2.36.1.windows.1.14.gc5ca16f7d9
cpu: x86_64
built from commit: c5ca16f7d97c11c75cc9a86461a91cedd593ce7b
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit? Windows Server 2016 64bit VM (Steps also performed on Windows 11 64 bit with same results)

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.14393]

• What options did you set as part of the installation? Or did you choose the
  defaults? defaults

type "C:\Program Files\Git\etc\install-options.txt"

Path Option: Cmd
SSH Option: OpenSSH
CURL Option: OpenSSL
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Enabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

No/Unknown

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

GitBash (SDK v1.0.8)

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

cd /usr/src/git
git fetch
git checkout main
make install
cd /usr/src/build/extra
git fetch
git checkout main
./installer/release.sh 2.36.1-test

Note: making the without building git works as intended. When building git it does not seem to generate all the files needed, or at least in comparison to the number of libexec files that existed before.

• What did you expect to occur after running these commands?

The generation of an installer using the newly compiled git from default branch main without any changes.

• What actually happened instead?

Installer fails to generate due to missing files which were not generated via make despite no apparent errors.

$ ./installer/release.sh 2.36.1-test
Generating release notes to be included in the installer ...
markdown is /usr/bin/markdown
Generating file list to be included in the installer ...
Launching Inno Setup compiler ...
Error on line 1213 in C:\git-sdk-64\usr\src\build-extra\installer\file-list.iss: Source file "C:\git-sdk-64\mingw64\libexec\git-core\git-add.exe" does not exist.
Compile aborted.
Could not make installer

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

** Build-Extra **

[dscho]

The most likely explanation is that the built-in commands (which are hard-linked into /mingw64/libexec/git-core/git-*.exe) are not built. That would be the case if you built Git using sdk build git (which the SDK installer does by default upon first startup) because that creates a config.mak that defines SKIP_DASHED_BUILT_INS=YesPlease.

Work-around: make -C /usr/src/git SKIP_DASHED_BUILT_INS= -j$(nproc) install.

I'll have to think how to best resolve this without needing a work-around.

[dscho]

This might present a fine opportunity to stop shipping Git for Windows with hard-linked executables for all the built-in commands.

Background: Due to its origin in the Linux ecosystem Git was originally designed as a set of "Unix-y tools" that were to be used in a set of shell scripts for the higher-level functionality. This turned out to be misguided very quickly, and more and more parts of Git were turned from POSIX shell scripts into commands implemented in C. To avoid a proliferation of ever larger executables, more and more commands were implemented as "built-in commands", i.e. the git executable included them and could execute them without spawning a separate executable. For backwards-compatibility, the so-called "dashed form" was still provided in the form of hard-linking the git executable to git-<command-name>, so that users could still call, say,git-add instead of git add. That was over a decade ago, though, and we tried a couple of times to drop those hard-linked executables but so far always failed "because backwards-compatibility!".

On the other hand, we already have a list of those built-in commands in /mingw64/share/git/builtins.txt, and we can easily teach build-extra's make-file-list.sh (which is the script that generates the list of files to include in the installer) to avoid listing these files if they do not exist.

[dscho]

@Amorik I implemented this in git-for-windows/build-extra#422. Could you please confirm that it fixes the issue on your end?

[Amorik]

@dscho Confirmed

executed the following steps on both systems. Note i did not do a complete test of the packaged installer, just ran it, stepped through and canceled.

cd /usr/src/git
make install
cd /usr/src/build-extra
git pull/422/head:gracefully-exclude-missing-dashed-executables
git checkout gracefully-exclude-missing-dashed-executables
./installer/release.sh 2.36.1-gracefully-exclude-missing-dashed-executables