git diff slow, git status fine - regression post 2.40.1

[clach04]

Summary

git diff is slow after Git-2.40.1, for a tiny checkout, git status is fine (so not #3318). Seen with Git-2.41.0.3-64-bit and latest (as of 2023-08-25) Git-2.42.0-64-bit. Not a problem in Git-2.26.2-64-bit and Git-2.40.1-64-bit (my current workaround). Boilerplate below.

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

C:\code\py\w2d_clean>C:\Programs\portablegit\Git-2.42.0-64-bit\cmd\git.exe --version --build-options
git version 2.42.0.windows.1
cpu: x86_64
built from commit: a2e49ec355ae22e74288fa74f50821f3cc1a1551
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

C:\code\py\w2d_clean>cmd.exe /c ver

Microsoft Windows [Version 10.0.19042.2965]

• What options did you set as part of the installation? Or did you choose the
  defaults?

I think defaults other than using Windows 10 OpenSSH.

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
> type "$env:USERPROFILE\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

C:\code\py\w2d_clean>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Enabled

newer 2.40.1 options

C:\code\py\w2d_clean>type "C:\Program Files\Git\etc\install-options.txt"
Editor Option: VIM
Custom Editor Path:
Default Branch Option: main
Path Option: Cmd
SSH Option: ExternalOpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Git Pull Behavior Option: Merge
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Enabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Nothing interesting! Super small checkout

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Doesn't matter, I've limited by test case below to CMD but bash is the same (and easier to time) BUT I don't want to confuse the issue with #4459

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git clone https://github.com/clach04/w2d.git w2d_clean
cd w2d_clean

# as of 2023-08-25 using commit:
C:\code\py\w2d_clean>git log -1
commit b833d29584be9905be4ed9af0902507c16a2ba34 (HEAD -> main, origin/main, origin/HEAD)
Author: clach04 <clach04@gmail.com>
Date:   Sat Aug 19 16:24:09 2023 -0700

    Doc epub reader for Mozilla Firefox

# I have a crappy Python script for timing commands (like a terrible version of Linux "time" command), you can instead simple use "time git diff" instead in Git Bash and see similar results

# NOTE using git portable for demo to show different times. Not shown, git status as it seems to be fine this is ONLY git diff - so different from https://github.com/git-for-windows/git/issues/3318

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py" "C:\Programs\portablegit\Git-2.42.0-64-bit\cmd\git.exe diff"
time taken: 0:00:03.626000 -- three seconds -- 3.626000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "C:\Programs\portablegit\Git-2.40.1-64-bit\cmd\git.exe diff"
time taken: 0:00:00.282000 -- 0 seconds -- 0.282000 secs

C:\code\py\w2d_clean>git version
git version 2.26.2.windows.1
C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py" "git diff"
time taken: 0:00:00.071000 -- 0 seconds -- 0.071000 secs

# there is some variation but in the same ball park for 2nd, 3rd, etc. runs

• What did you expect to occur after running these commands?

For this repo essentially instant (empty) response

• What actually happened instead?

3-4 second delay on git diff

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

It is not specific to a repo, but you are welcome to use this super small repo for testing

https://github.com/clach04/w2d.git

[dscho]

@clach04 could you see whether git -p status is quick or slow?

[clach04]

@dscho Interesting, "bad versions" are slow. Good versions fast/quick. I.e. slow diff == slow -p status.

C:\code\py\w2d_clean>git version
git version 2.40.1.windows.1

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.122000 -- 0 seconds -- 0.122000 secs


C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "C:\Programs\portablegit\Git-2.42.0-64-bit\cmd\git.exe -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:04.091000 -- four seconds -- 4.091000 secs

[clach04]

Oh crap, this is looking like #4459 after all!

Thanks @dscho .

I tried using the windows more:

PAGER=C:\Windows\System32\more.com
git config --global core.pager C:\Windows\System32\more.com

as a potential work around with no luck :-(

I also tried using less from the faster 40.1 release, again no success. I guess there is more msys usage going on under the covers:

C:\code\py\w2d_clean>set PAGER=C:\Programs\portablegit\Git-2.40.1-64-bit\usr\bin\less.exe

C:\code\py\w2d_clean>git config --global core.pager C:\Programs\portablegit\Git-2.40.1-64-bit\usr\bin\less.exe

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "C:\Programs\portablegit\Git-2.42.0-64-bit\cmd\git.exe -p status"
C:\Programs\portablegit\Git-2.40.1-64-bit\usr\bin\less.exe: line 1: C:ProgramsportablegitGit-2.40.1-64-bitusrbinless.exe: command not found
time taken: 0:00:03.899000 -- three seconds -- 3.899000 secs

[clach04]

I'll check this again on next release now #4459 is fixed.

[dscho]

I'll check this again on next release now #4459 is fixed.

That's too late. It would be better to check with the build artifact of https://github.com/git-for-windows/msys2-runtime/actions/runs/6003432176#artifacts to help whip out a solid next release.

[clach04]

@dscho 👍 I've upgraded to https://github.com/git-for-windows/git/releases/tag/v2.42.0.windows.1

Saw same performance/slow-down.

Grabbed zip from https://github.com/git-for-windows/msys2-runtime/actions/runs/6003432176#artifacts - extracted into Git directory, performance back to normal:

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.101000 -- 0 seconds -- 0.101000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.117000 -- 0 seconds -- 0.117000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.117000 -- 0 seconds -- 0.117000 secs

C:\code\py\w2d_clean>git version
git version 2.40.1.windows.1

C:\code\py\w2d_clean>git version
git version 2.42.0.windows.1

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:02.578000 -- two seconds -- 2.578000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:02.123000 -- two seconds -- 2.123000 secs

C:\code\py\w2d_clean>git version
git version 2.42.0.windows.1

C:\code\py\w2d_clean>echo extracted usr dir
extracted usr dir

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:01.230000 -- one second -- 1.230000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.102000 -- 0 seconds -- 0.102000 secs

C:\code\py\w2d_clean>py -2 "C:\Users\clach04\OneDrive - Actian Software\presentations\pre_2014\ukiua_2012\benchmark_fastloading\mytimer.py"  "git -p status"
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
time taken: 0:00:00.116000 -- 0 seconds -- 0.116000 secs

[clach04]

I think it is reasonable to state that this is another manifestation of #4459 - I'll leave open and let someone else make decision about closing.

[dscho]

@clach04 excellent. This gives me a lot of confidence that the problem is fundamentally fixed.

Will close this here ticket as a duplicate of #4459 (as we now know in hindsight). I want to point out that I am grateful for your work in this here ticket.

[clach04]

Final validation. Tested fine with (latest) git version 2.42.0.windows.2 from https://github.com/git-for-windows/git/releases/tag/v2.42.0.windows.2 :-)
Many thanks to @dscho and @rglidden!