"call depth overflow" exception in Powershell scripts on Mac

[mpeneva]

Some of of my PowerShell scripts that I run on Mac failed with the following exception:
"The script failed due to call depth overflow."
After some investigation, I came to the conclusion that the allow call depth in PowerShell v6.0 alpha,
is smaller than the defined in PowerShell v5.0 (which I find is 100).
Could you change or increase in PowerShell for Unix that fixed call depth to be at least 1000?

[iSazonov]

Can you reproduce with latest version - Beta.4?
If so please add repo steps and more description.

[mpeneva]

Hi and thanks for your response. I've updated the PS to version 4.0 beta, however it occurred a new issue, so we started looking at it with one of my colleagues, so once we're ready with that, I would try the above described issue with the beta version of PS.

[iSazonov]

Could you please add simple repo steps?

[mpeneva]

Sure, I would , but currently there is a 'local' problem, that's not connected at all with the PS version.

[it-praktyk]

I've observed the same issue what you can easily reproduce.

Please clone the Pester branch Dev4-1_PSCore_compatibility, check out to the commit 89344732785b862c4c0fcc72050290df6fce9e5f and run tests from the file Functions/Assertions/Be.Tests.ps1.

The test result

<OUTPUT_PARTIALL_OMMITED>

    [-] throws exception when self-imposed recursion limit is reached 592ms
      Expected: the expression to throw an exception with message {recursion depth limit}, an exception was raised, message was {The script failed due to call depth overflow.}
          from <FOLDER_PATH>/Pester/Functions/Assertions/Be.Tests.ps1:99 char:21
          +             { $a1 | Should be $a2 } | Should throw 'recursion depth l ...
          +                     ~~~~~~~~~~~~~
      99:             { $a1 | Should be $a2 } | Should throw 'recursion depth limit'
      at Invoke-LegacyAssertion, <FOLDER_PATH>/Pester/Functions/Assertions/Should.ps1: line 190
      at <ScriptBlock>, <FOLDER_PATH>/Pester/Functions/Assertions/Be.Tests.ps1: line 99

<OUTPUT_PARTIALL_OMMITED>

The failed test is the one that should prevent this kind of situation - it was introduced to Pester here.

It works correctly for Windows (2.0 - 6.0.0-beta.8) and Linux (6.0.0-beta.8). The issue was the first reported on PSCore 6.0.0-beta.5.

[it-praktyk]

Today I've tested it on macOS and PowerShell Core v.6.0.0-beta.9 again. The issue still exists.

[it-praktyk]

Under the bug reproducing please use exact the code state - the link provided above. Later I've added a condition to don't run the failing test under macOS.

[SteveL-MSFT]

This seems to be an issue with coreclr.

function recurse([int]$i) { $i; recurse ($i+1) }
recurse 0

On Mac, this fails after ~100 iterations
On Windows, ~1400
On Linux, ~2800

[iSazonov]

Can repo on C#?

[Jaykul]

Are you hitting the stack limit? Try checking it with ulimit -Sa, I understand it's smaller on Mac OS than Linux ...

[SteveL-MSFT]

On both my MacBook Pro and Ubuntu 16.04, it's set to 8192k

[alekol]

I tried increasing the stack size to 65k, made no difference..

Is anyone looking into this?

[iSazonov]

Analyze of the stack limitations on Russian https://habrahabr.ru/post/320970/

I think the code is here. It is under #if CORECLR. I don't know why we skipped this and don't remove .Net Core 1.0 code. I believe we can easily fix this.
/cc @SteveL-MSFT Sorry I'm on vacation and I can't do it on the week.

[SteveL-MSFT]

@iSazonov enjoy your vacation. Let me try a private patch to see if this fixes this by providing the maxstack parameter.

[SteveL-MSFT]

Explicitly setting the MaxStack parameter when creating a new thread didn't have any effect even when set to 100MB.

[iSazonov]

I found that CoreCLR has a trace but I don't know how turn in it https://github.com/dotnet/coreclr/blob/c440335be80ee0762856d0be6e91ec3ea2f90504/src/pal/src/thread/thread.cpp#L663

[iSazonov]

MacOS set 8MB stack size for process but 512KB for subsequent threads.

Windows default is 1 MB ("The default stack reservation size used by the linker is 1 MB.")

[iSazonov]

Windows PowerShell Default for local pipeline thread stack size is 10 MB.

[janvorli]

This has nothing to do with stack size. Powershell has implicit limit of max 100 levels of recursion no matter how much stack is left. See the following article, the "PowerShell's Call Depth Limit" section:
https://weblogs.asp.net/jongalloway/working-around-a-powershell-call-depth-disaster-with-trampolines

[janvorli]

I am sorry, I've not realized that the limit was for powershell 1.0. So please disregard my comment above.

[iSazonov]

Currently we set 10 MB stack size for local pipeline threads on all platforms.
We can change the value based on feedback in future.