git rebase bails out with sh.exe.stackdump file (Exception: STATUS_STACK_OVERFLOW)

[iafan]

• I was not able to find an open
  or closed issue
  matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

64-bit

$ git --version --build-options
git version 2.10.0.windows.1
sizeof-long: 4
machine: x86_64

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver
Microsoft Windows [Version 10.0.14393]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt
Path Option: CmdTools
SSH Option: OpenSSH
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: ConHost
Performance Tweaks FSCache: Enabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

This started happening after I upgraded to Git-2.10

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

  CMD

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

$ git rebase

• What did you expect to occur after running these commands?

  Rebase should have happened without any issues.

• What actually happened instead?

  Rebase only prints First, rewinding head to replay your work on top of it... and bails out. A sh.exe.stackdump file created in the current directory with the following contents:

Exception: STATUS_STACK_OVERFLOW at rip=7FF99CA6ACB7
rax=0000000000002058 rbx=0000000000763E30 rcx=00007FF99CA77DF0
rdx=0000000000000001 rsi=0000000180010018 rdi=00000000FFFFB078
r8 =00000001802A8940 r9 =00000000FFFFAE68 r10=00000000FFFF8000
r11=00000000FFE03830 r12=00000000FFFFAEF0 r13=00000000000001A4
r14=00000001802A8940 r15=0000000000000000
rbp=00000001802A8940 rsp=00000000FFFFADE0
program=C:\Program Files\Git\usr\bin\sh.exe, pid 6864, thread unknown (0x283C)
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
001802A8940  7FF99CA6ACB7 (7FF99CA77DF0, 00000000001, 001802A8940, 000FFFFAE68)
001802A8940  7FF99CA676DE (00000000001, 001802A8940, 000FFFFAE68, 00000000000)
001802A8940  7FF99CA5F79E (000FFFFAF50, 000FFFFB070, 00000000420, 00000000001)
000FFFFAF50  001800BEA3B (000FFFFB010, 00000000030, 00000000000, 00000000000)
000FFFFB090  001800BF5E5 (0060007DE00, 00000000001, 0010040FDFE, 00000000001)
00000000000  0018012F44B (0060007DE00, 00000000001, 0010040FDFE, 00000000001)
00000000000  00002B7E458 (00000000001, 0010040FDFE, 00000000001, 00000080002)
00000000000  001004E172E (0010040FDFE, 00000000001, 00000080002, 00000080000)
00000000000  0060007DE00 (00000000001, 00000080002, 00000080000, 00000000011)
00000000000  00000000001 (00000080002, 00000080000, 00000000011, 00000000000)
00000000000  0010040FDFE (000FFFFFF00, 001004D6701, 000005F6701, 001005E8C88)
00000000000  001004D67B4 (001004D6701, 000005F6701, 001005E8C88, 00100414195)
00000000000  0060007DE00 (000005F6701, 001005E8C88, 00100414195, 00000000018)
00000000000  000FFFFFF00 (001005E8C88, 00100414195, 00000000018, 00180339748)
00000000000  001004D6701 (00100414195, 00000000018, 00180339748, 00000000018)
00000000000  000005F6701 (00000000018, 00180339748, 00000000018, 00180338898)
End of stack trace (more stack frames may be present)

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

  This happens on all local checkouts.

[iafan]

Just tried reinstalling Git (removing, rebooting, installing) with the same settings. The problem persists.

Trace information:

$ set GIT_TRACE=1
$ git rebase
21:40:30.775061 git.c:564               trace: exec: 'git-rebase'
21:40:30.775061 run-command.c:336       trace: run_command: 'git-rebase'
21:40:31.110060 git.c:350               trace: built-in: git 'rev-parse' '--git-dir'
21:40:31.197208 git.c:350               trace: built-in: git 'rev-parse' '--git-path' 'objects'
21:40:31.254250 git.c:350               trace: built-in: git 'rev-parse' '--is-bare-repository'
21:40:31.308488 git.c:350               trace: built-in: git 'rev-parse' '--show-toplevel'
21:40:31.382200 git.c:350               trace: built-in: git 'config' '--bool' 'rebase.stat'
21:40:31.451578 git.c:350               trace: built-in: git 'config' '--bool' 'rebase.autostash'
21:40:31.508469 git.c:350               trace: built-in: git 'config' '--bool' 'rebase.autosquash'
21:40:31.563977 git.c:350               trace: built-in: git 'config' '--bool' 'commit.gpgsign'
21:40:31.707209 git.c:350               trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/origin/master^0'
21:40:31.794205 git.c:350               trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/origin/master^0'
21:40:31.864732 git.c:350               trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
21:40:31.951166 git.c:350               trace: built-in: git 'rev-parse' '--verify' 'HEAD'
21:40:32.013714 git.c:350               trace: built-in: git 'merge-base' '--fork-point' 'refs/remotes/origin/master' 'HEAD'
21:40:32.095270 git.c:350               trace: built-in: git 'rev-parse' '--verify' 'HEAD'
21:40:32.155528 git.c:350               trace: built-in: git 'update-index' '-q' '--ignore-submodules' '--refresh'
21:40:32.237313 git.c:350               trace: built-in: git 'diff-files' '--quiet' '--ignore-submodules'
21:40:32.318860 git.c:350               trace: built-in: git 'diff-index' '--cached' '--quiet' '--ignore-submodules' 'HEAD' '--'
21:40:32.386406 git.c:350               trace: built-in: git 'merge-base' '8c1a54068b5d3653b5330381589ad108a8f8bd8a' '8c1a54068b5d3653b5330381589ad108a8f8bd8a'
21:40:32.451433 git.c:350               trace: built-in: git 'rev-parse' '--git-path' 'hooks/pre-rebase'
First, rewinding head to replay your work on top of it...
21:40:32.528255 git.c:350               trace: built-in: git 'checkout' '-q' '8c1a54068b5d3653b5330381589ad108a8f8bd8a^0'
21:40:32.643012 git.c:350               trace: built-in: git 'update-ref' 'ORIG_HEAD' '8c1a54068b5d3653b5330381589ad108a8f8bd8a'

21:40:32.948747 git.c:350               trace: built-in: git 'rev-parse' 'HEAD'
21:40:33.008510 git.c:350               trace: built-in: git 'update-ref' '-m' 'rebase finished: refs/heads/master onto 8c1a54068b5d3653b5330381589ad108a8f8bd8a' 'refs/heads/master' '8c1a54068b5d3653b5330381589ad108a8f8bd8a' '8c1a54068b5d3653b5330381589ad108a8f8bd8a'
21:40:33.067125 git.c:350               trace: built-in: git 'symbolic-ref' '-m' 'rebase finished: returning to refs/heads/master' 'HEAD' 'refs/heads/master'
21:40:33.126159 git.c:350               trace: built-in: git 'gc' '--auto'

[iafan]

git gc --auto works fine by itself, so there's something after this step that causes the crash.

[iafan]

Before upgrading to Git 2.10, I was using ~1 year-old version, but didn't remember what it was. I started installing older versions to determine where the issue was introduced. Here's what I've got:

• v2.9.3.windows.2 — same issue
• v2.9.2.windows.1 — same issue
• v2.8.4.windows.1 — same issue
• v2.7.3.windows.1 — same issue
• v2.7.1.windows.2 — same issue
• v2.7.1.windows.1 — same issue
• v2.7.0.windows.2 — same issue
• v2.7.0.windows.1 — no issue
• v2.6.0.windows.1 — no issue

Looks like something have changed between v2.7.0.windows.1 and v2.7.0.windows.2 that causes the issue.

[PhilipOakley]

Is the repo you are using for the rebase public?

Can you preserve its current state, so at least the test case is still available.

The fact that, for your repo/rebase, the issue goes back so far, suggests that it is a special corner case that requires some aspect of your repo to trigger the stack dump (folk are quick to report such effects normally..)

Additionally the change v2.7.0.windows.1 to w.2 is internal to the windows fixup, so those few changes should be easy to locate, which may show more light on the possible issues.

Philip

PS, dscho, the maintainer, is currently on an extended vacation for a few weeks.

----- Original Message -----
From: Igor Afanasyev
To: git-for-windows/git
Sent: Sunday, September 18, 2016 6:41 AM
Subject: Re: [git-for-windows/git] git rebase bails out with sh.exe.stackdump file (Exception: STATUS_STACK_OVERFLOW) (#889)

Before upgrading to Git 2.10, I was using ~1 year-old version, but didn't remember what it was. I started installing older versions to determine where the issue was introduced. Here's what I've got:

a.. v2.9.3.windows.2 — same issue 
b.. v2.9.2.windows.1 — same issue 
c.. v2.8.4.windows.1 — same issue 
d.. v2.7.3.windows.1 — same issue 
e.. v2.7.1.windows.2 — same issue 
f.. v2.7.1.windows.1 — same issue 
g.. v2.7.0.windows.2 — same issue 
h.. v2.7.0.windows.1 — no issue 
i.. v2.6.0.windows.1 — no issue 

Looks like something have changed between v2.7.0.windows.1 and v2.7.0.windows.2 that causes the issue.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[iafan]

Yes, the repos I tried this on are public (and since this behavior occurred for me for every repo I tried, I believe this is not repo specific). Just as an example, I tried doing a fresh clone of this small repo of mine, ran git rebase immediately after that, and got the same crash.

So I guess this crash is specific to my system rather than to the repo. Not sure how to help debug it further. Do you happen to know how sh.exe is run during rebase and with what parameters?

[PhilipOakley]

The easy way to debug further is via the GfW SDK (wooo). It's at https://git-for-windows.github.io/#contribute

It should take (depending on your m/c speed and connection speed) about 30 mins to download the full source, compile and install the latest version, allowing you to try out the very latest version, reverting the two main changes in that V2.7.0(2) https://github.com/git-for-windows/git/releases/tag/v2.7.0.windows.2 of the ASLR/DEP and the 'pull--rebase' fixes, and see if any of those steps have any impact.

Check what version (fixes & patches) of W10 you have just in case. There were some issues on the list where folks who had advance releases had issue that MS then fixed.

There has also been the usual dll adress space layout issues (the MS 'rebase' of the dlls [which is totally different from the 'git rebase'] often fixes that).

Philip
----- Original Message -----
From: Igor Afanasyev
To: git-for-windows/git
Cc: Philip Oakley ; Comment
Sent: Sunday, September 18, 2016 5:35 PM
Subject: Re: [git-for-windows/git] git rebase bails out with sh.exe.stackdump file (Exception: STATUS_STACK_OVERFLOW) (#889)

Yes, the repos I tried this on are public (and since this behavior occurred for me for every repo I tried, I believe this is not repo specific). Just as an example, I tried doing a fresh clone of this small repo of mine, ran git rebase immediately after that, and got the same crash.

So I guess this crash is specific to my system rather than to the repo. Not sure how to help debug it further. Do you happen to know how sh.exe is run during rebase and with what parameters?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[dscho]

This issue looks like a .dll base address problem (such problems are rare on 64-bit setups, though). It is also possible that the update of the MSYS2 runtime (<Git>\usr\bin\msys-2.0.dll) caused it...

Could you try out v2.10.1? If that still fails, could you install the portable versions 2.7.0 and 2.7.0.2 side by side and see whether the problem is still reproducible with those? If so, could you replace the latter's msys-2.0.dll with the former's?

[iafan]

Could you try out v2.10.1?

The issue is still reproducible there.

If that still fails, could you install the portable versions 2.7.0 and 2.7.0.2 side by side and see whether the problem is still reproducible with those?

Yes, the issue is reproducible in portable version 2.7.0.2 and is not reproducible in portable 2.7.0.

If so, could you replace the latter's msys-2.0.dll with the former's?

After copying over usr\bin\msys-2.0.dll from 2.7.0 to 2.7.0.2 the problem is no longer reproducible in 2.7.0.2.

[dscho]

@iafan thanks for testing this.

The bigger problem now is: the changes between 2.7.0's and 2.7.0.2's runtime are quite a bit extensive. So the next step is to whittle down the search scope even further, preferably by bisecting.

So here is my question: do you time to invest in this search (if so, I will describe what to do after installing the SDK?

[dscho]

@iafan maybe I could ask you to test in v2.11.0(3)?

[iafan]

@dscho 2.11.0.windows.3 seems to be working fine. Thanks for having this fixed!

[kalianto]

I just installed v2.12.2 64 bit and got the same error when rebasing

Exception: STATUS_STACK_OVERFLOW at rip=7FF91271D018
rax=0000000000001AA8 rbx=00000000007597E0 rcx=0000000180010018
rdx=0000000000743510 rsi=0000000000743510 rdi=00000000FFFFAEF0
r8 =0000000000000000 r9 =00000000000026C0 r10=00000000FFFF9000
r11=00000000FFE037D0 r12=0000000000000000 r13=0000000000759A40
r14=00000000000026C0 r15=0000000000000420
rbp=00000000FFFF91F0 rsp=00000000FFFFAB80
program=C:\Program Files\Git\usr\bin\sh.exe, pid 9920, thread unknown (0x2340)
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
000FFFF91F0  7FF91271D018 (000000026C0, 00000759A40, 00000000000, 000FFFFAEF0)
000FFFF91F0  7FF9126FF17F (000FFFFACE0, 000FFFFFFFF, 00000000000, 000007597E0)
000FFFFACE0  7FF9126FF663 (000FFFFADB0, 000000001B4, 00000000000, 00180277C00)
00000000420  7FF913139FF6 (00000000001, 00000000000, 000FFFFAE90, 00100000001)
00000000420  7FF91627BF13 (00000000020, 00000000000, 000FFFFB010, 00000000001)
00000000420  001800AA082 (000FFFFAFB0, 00000000000, 7FF900000000, 00000000000)
000FFFFB030  001800AAC25 (00100410FBB, 00000000001, 00600081240, 001004E6740)
000FFFFB230  0018011A60B (00100410FBB, 00000000001, 00600081240, 001004E6740)
000FFFFB230  0000140E458 (00100410FBB, 00000000001, 00600081240, 001004E6740)
End of stack trace

So it starts with First, rewinding head to replay your work on top of it...
then it produces sh.exe.stackdump file and the content of the file is as above.

I have Comodo Cloud AntiVirus installed, it blocks git-bash.exe and perhaps more.