ArduPilot's use of submodules and recommended workflow hose the git tree when rebasing on top of master

[Ryanf55]

Bug report

Issue details

When using git submodules, when they are moved around, it causes major issues when switching branches.

Version

• Start with HEAD at 5c1ed96 then try rebase on f67d895

• git version: 2.34.1

• OS of Ubuntu 22.04.3

Platform
[x ] All
[ ] AntennaTracker
[ ] Copter
[ ] Plane
[ ] Rover
[ ] Submarine

Steps to reproduce

1. Start with an empty directory
2. Clone my repo as origin ryanf55/ardupilot with --recurse-submodules at hash 5c1ed96
3. Change directory into ardupilot
4. Add an "upstream" remote of ardupilot/ardupilot
5. Fetch upstream
6. Rebase origin on upstream
7. Congrats, the work tree is hosed

Attempted recovery

• git submodule deinit --all --force
• git clean -dxf
• git reset --hard origin/master
• ./Tools/gittools/submodule-sync.sh
• git reset --hard upstream/master
• $ git reset --hard upstream/master

fatal: not a git repository: ../../.git/modules/modules/lwip
error: Submodule 'modules/lwip' could not be updated.
error: Submodule 'modules/lwip' cannot checkout new HEAD.
fatal: Could not reset index file to revision 'upstream/master'.

Logs

Full logs to reproduce this are attached.
log.txt

The exact error occurs here:

ryan@B650-970:~/Dev/tmp_ws/ardupilot$ git rebase upstream/master
fatal: not a git repository: ../../.git/modules/modules/lwip
fatal: could not reset submodule index
error: could not detach HEAD

Recommendations

1. Add documentation to the wiki on how to switch branches and rebase without error
   OR
2. Switch to a different version control tool for dependencies such as Conan, CPM, VCS, Google Repo

References

• https://git-scm.com/book/en/v2/Git-Tools-Submodules#Issues-with-Submodules
• https://stackoverflow.com/questions/45215094/git-giving-warning-unable-to-rmdir/64724218#64724218

[tridge]

I just followed your list of steps and had no issues. All steps worked fine
here is a log of all steps:
http://uav.tridgell.net/tmp/git-test.txt

[tridge]

@Ryanf55 I've now reproduced. The key step which you didn't call out in the description is "git config submodule.recurse true". If you don't do that then it all works fine.
I've never set that config option myself, and just use --recurse when I want that (I often don't)

[Ryanf55]

@Ryanf55 I've now reproduced. The key step which you didn't call out in the description is "git config submodule.recurse true". If you don't do that then it all works fine. I've never set that config option myself, and just use --recurse when I want that (I often don't)

Thanks for looking at this and reproducing the issue!

I'll create a Wiki PR to recommend turning off the submodule.recurse locally in the ArduPilot repo. ^