builtins.fetchGit fails with .gitmodules entries that aren't submodules

[alyssais]

Describe the bug

In the situation where a git submodule used to exist, has been moved into the main repo, but not removed from .gitmodules, Nix is unable to fetchGit with submodules. git clone is able to handle this fine, because .gitmodules is not the source of truth for what submodules exist — it's only used as a lookup table for where to fetch unpopulated submodules. I think the right thing for Nix to do here is look through the git tree, and only fill in submodules that are actually submodules, ignoring other .gitmodules entries.

Steps To Reproduce

nix-repl> builtins.fetchGit { url = https://chromium.googlesource.com/chromiumos/platform/crosvm; ref = "release-R124-15823.B"; submodules = true; }
error:
       … while calling the 'fetchGit' builtin
         at «string»:1:1:
            1| builtins.fetchGit { url = https://chromium.googlesource.com/chromiumos/platform/crosvm; ref = "release-R124-15823.B"; submodules = true; }
             | ^

       … while fetching the input 'git+https://chromium.googlesource.com/chromiumos/platform/crosvm?ref=release-R124-15823.B&submodules=1'

       error: '«unknown»/third_party/vmm_vhost' is not a submodule

Expected behavior

Works.

nix-env --version output

nix-env (Nix) 2.22.0

Additional context

Add any other context about the problem here.

Priorities

Add 👍 to issues you find important.

[fricklerhandwerk]

Triage in Nix maintainers meeting:

• @Ericson2314: Yes, this is indeed a bug. .gitmodules must not be the source of truth. @roberth and I tested the opposite thing where we kept the submodule and removed the .gitmodules entry.

  Currently, a submodule is always replaced with an empty directory, and a "bind mount" SourceAccessor is used to overlay the missing submodules based on .gitmodules. That is indeed wrong. Instead, every time a submodule (child commit object) is found, we should call some sort of callback to do the right thing, which forces the case to be handled from the proper source of truth.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-05-22-nix-team-meeting-minutes-147/45835/1