Asserting MockCalled causes an exception (from within Test-ParameterFilter)

[Glober777]

General summary of the issue

I'm testing a module where a member cmdlet calls Start-Job. In my test, I'm mocking Start-Job in the test (which works fine) however my tests fail on Assert-MockCalled (as well as on Should -Invoke ...) throwing an exception:

ParameterBindingValidationException: Cannot bind argument to parameter 'SessionState' because it is null.

Describe your environment

Pester version : 5.1.1 C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
PowerShell version : 7.1.1
OS version : Microsoft Windows NT 10.0.18363.0

Steps to reproduce

In my simplified repro project folder I have two subfolders:

• MyModule
• Tests

In MyModule folder I have a MyModule.psm1 file with the following:

function Invoke-SomeAction {
    [CmdletBinding()]
    param (

    )

    $TheJob = Start-Job -InitializationScript {"Some Init stuff"} -ScriptBlock {"The scriptblock"}

    $TheJob | Wait-Job | receive-Job

    Remove-Job -Job $TheJob
}


Export-ModuleMember Invoke-SomeAction

In Tests folder I have the following (Repro.Tests.ps1):

BeforeDiscovery {
    $ModuleName = 'MyModule'
    $ModuleManifestName = "$ModuleName.psm1"
    $ModulePath = ("$PSScriptRoot\..\$ModuleName" | Resolve-Path).Path
    $ModuleManifestPath = "$ModulePath\$ModuleManifestName"

    If (!(Get-Module $ModuleName)) {
        $Module = Import-Module $ModuleManifestPath -PassThru -ErrorAction Stop
    }
}

Describe "Repro for the assertion issue"  -ForEach @(
    @{
        ModuleName = $ModuleName
    }
) {
    BeforeAll {
        $StartJobMock = Get-Command Start-Job

        Mock Start-Job -MockWith {
            & $StartJobMock -ScriptBlock {"Mocked scriptblock"}
        } -ModuleName $_.ModuleName

    }

    It "Assertion works" {
        Invoke-SomeAction | Should -BeExactly 'Mocked scriptblock'
        # Should -Invoke -CommandName Start-Job -Times 1 -Scope It
        Assert-MockCalled -CommandName Start-Job -Times 1 -Scope It
    }
}

Now, every time I either run Invoke-Pester or press Run Tests codelens I constantly get the abovementioned exception

Expected Behavior

Test should succeed

Current Behavior

Test fails with an exception:

Exception

Exception             : 
    Type              : System.Management.Automation.ParameterBindingValidationException
    Message           : Cannot bind argument to parameter 'SessionState' because it is null.
    ParameterName     : SessionState
    ParameterType     : System.Management.Automation.SessionState
    ErrorId           : ParameterArgumentValidationErrorNullNotAllowed
    Line              : 9565
    Offset            : 34
    CommandInvocation : 
        MyCommand        : Test-ParameterFilter
        BoundParameters  : 
            Comparer : System.OrdinalIgnoreCaseComparer
            Count    : 2
            Keys     : 
                Length : 11

                Length : 8
            Values   : 
                File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                Module        : Pester
                StartPosition : System.Management.Automation.PSToken
                Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                Ast           : { $True }


            SyncRoot : 
                Comparer : System.OrdinalIgnoreCaseComparer
                Count    : 2
                Keys     : 
                    Length : 11

                    Length : 8
                Values   : 
                    File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                    Module        : Pester
                    StartPosition : System.Management.Automation.PSToken
                    Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                    Ast           : { $True }


                SyncRoot : 
                    Comparer : System.OrdinalIgnoreCaseComparer
                    Count    : 2
                    Keys     : 
                        Length : 11

                        Length : 8
                    Values   : 
                        File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                        Module        : Pester
                        StartPosition : System.Management.Automation.PSToken
                        Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                        Ast           : { $True }


                    SyncRoot : 
                        Comparer : System.OrdinalIgnoreCaseComparer
                        Count    : 2
                        Keys     : 
                            Length : 11

                            Length : 8
                        Values   : 
                            File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                            Module        : Pester
                            StartPosition : System.Management.Automation.PSToken
                            Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                            Ast           : { $True }


                        SyncRoot : 
                            Comparer : System.OrdinalIgnoreCaseComparer
                            Count    : 2
                            Keys     : 
                                Length : 11

                                Length : 8
                            Values   : 
                                File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                                Module        : Pester
                                StartPosition : System.Management.Automation.PSToken
                                Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                                Ast           : { $True }


                            SyncRoot : 
                                Comparer : System.OrdinalIgnoreCaseComparer
                                Count    : 2
                                Keys     : 
                                    Length : 11

                                    Length : 8
                                Values   : 
                                    File          : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
                                    Module        : Pester
                                    StartPosition : System.Management.Automation.PSToken
                                    Id            : 0e832cdf-5fd7-410f-b3df-33f96064978f
                                    Ast           : { $True }


                                SyncRoot : 
                                    Comparer : System.OrdinalIgnoreCaseComparer
                                    Count    : 2
                                    Keys     : …
                                    Values   : …
                                    SyncRoot : …
        ScriptLineNumber : 9565
        OffsetInLine     : 13
        HistoryId        : 299
        ScriptName       : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
        Line             : if (Test-ParameterFilter @params) {

        PositionMessage  : At C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1:9565 char:13
                           +         if (Test-ParameterFilter @params) {
                           +             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        PSScriptRoot     : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1
        PSCommandPath    : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
        InvocationName   : Test-ParameterFilter
        PipelineLength   : 1
        PipelinePosition : 1
        CommandOrigin    : Internal
    ErrorRecord       : 
        Exception             : 
            Type    : System.Management.Automation.ParentContainsErrorRecordException
            Message : Cannot bind argument to parameter 'SessionState' because it is null.
            HResult : -2146233087
        CategoryInfo          : InvalidData: (:) [Test-ParameterFilter], ParentContainsErrorRecordException
        FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Test-ParameterFilter
        InvocationInfo        : 
            MyCommand        : Test-ParameterFilter
            ScriptLineNumber : 9565
            OffsetInLine     : 34
            HistoryId        : 299
            ScriptName       : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
            Line             : if (Test-ParameterFilter @params) {

            PositionMessage  : At C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1:9565 char:34
                               +         if (Test-ParameterFilter @params) {
                               +                                  ~~~~~~~
            PSScriptRoot     : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1
            PSCommandPath    : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
            CommandOrigin    : Internal
        ScriptStackTrace      : at Should-InvokeInternal, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 9565
                                at Should-Invoke, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 13118
                                at Invoke-Assertion, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 7357
                                at Should<End>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 7307
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 12823
                                at Assert-MockCalled, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 12827
                                at <ScriptBlock>, C:\Users\user1\OneDrive - PPD\Scripts\Prototyping\MockStartJob\Tests\Repro.tests.ps1: line 29
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                                at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2046
                                at Invoke-TestItem, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1142
                                at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 793
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 848
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                                at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2049
                                at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 893
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 848
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                                at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                                at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2049
                                at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 893
                                at Run-Test, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1601
                                at Invoke-Test, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2387
                                at Invoke-Pester<End>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 4839
                                at <ScriptBlock>, <No file>: line 1
    TargetSite        : 
        Name          : CheckActionPreference
        DeclaringType : System.Management.Automation.ExceptionHandlingOps, System.Management.Automation, Version=7.1.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
        MemberType    : Method
        Module        : System.Management.Automation.dll
    StackTrace        : 
   at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
   at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)
   at System.Management.Automation.PSScriptCmdlet.RunClause(Action`1 clause, Object dollarUnderbar, Object inputToProcess)
   at System.Management.Automation.CommandProcessorBase.Complete()
    Data              : System.Collections.ListDictionaryInternal
    Source            : System.Management.Automation
    HResult           : -2146233087
CategoryInfo          : InvalidData: (:) [Test-ParameterFilter], ParameterBindingValidationException
FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Test-ParameterFilter
InvocationInfo        : 
    MyCommand        : Test-ParameterFilter
    ScriptLineNumber : 9565
    OffsetInLine     : 34
    HistoryId        : 299
    ScriptName       : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
    Line             : if (Test-ParameterFilter @params) {

    PositionMessage  : At C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1:9565 char:34
                       +         if (Test-ParameterFilter @params) {
                       +                                  ~~~~~~~
    PSScriptRoot     : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1
    PSCommandPath    : C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1
    CommandOrigin    : Internal
ScriptStackTrace      : at Should-InvokeInternal, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 9565
                        at Should-Invoke, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 13118
                        at Invoke-Assertion, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 7357
                        at Should<End>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 7307
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 12823
                        at Assert-MockCalled, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 12827
                        at <ScriptBlock>, C:\Users\user1\OneDrive - PPD\Scripts\Prototyping\MockStartJob\Tests\Repro.tests.ps1: line 29
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                        at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2046
                        at Invoke-TestItem, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1142
                        at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 793
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 848
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                        at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2049
                        at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 893
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 848
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1924
                        at <ScriptBlock>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1885
                        at Invoke-ScriptBlock, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2049
                        at Invoke-Block, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 893
                        at Run-Test, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 1601
                        at Invoke-Test, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 2387
                        at Invoke-Pester<End>, C:\Users\user1\Documents\PowerShell\Modules\Pester\5.1.1\Pester.psm1: line 4839
                        at <ScriptBlock>, <No file>: line 1

Possible Solution? (optional)

None

[JustinGrote]

Here's a workaround for now I'm using:

[Int]$SCRIPT:RemoveReleaseInvokeCount = 0
Mock -ModuleName Press Remove-GithubRelease { $SCRIPT:RemoveReleaseInvokeCount++ }
...
...
...
$SCRIPT:RemoveReleaseInvokeCount | Should -Be $RemoveCount

[fflaten]

Have you tried specifying -ModuleName in the Should-assertion? You both create your mocks inside your module's scope, while asserting if a mock with the same name outside the module's scope was called.

@Glober777 Try: Should -Invoke -CommandName Start-Job -ModuleName $_.ModuleName -Times 1 -Scope It
@JustinGrote: Try Should -Invoke 'Remove-GithubRelease' -ModuleName Press -Exactly -Times $RemoveCount

[fflaten]

@JustinGrote: Is the original Remove-GithubRelease by any chance a function defined outside of the Press-module?

I believe this issue occurs while creating mocks inside YourModule for functions that originally exists somewhere else, just like the built in Start-Job in OP. If you mock an internal function inside your module using -ModuleName YourModule but forget to specify the ModuleName-parameter in Should-Invoke, the assertion would've thrown CommandNotFoundException as expected since the internal function never existed outside the module-scope (except for possible name-conflict with a different module's public function). This would've made it easier for you to identify the inconsistent usage of -ModuleName in setup/mock vs assertion.

However when the original command is external to the module where your mock is injected and you forget -ModuleName YourModule in Should -Invoke, the current logic gets confused. The assertion resolves the commandname to the original command that's visible outside of the module-scope, while the mock CallHistory lists the mock invocation. This mismatch causes Test-ParameterFilter to be executed on the mock invocation log-entry while using the ContextInfo of the original command which fails due to missing custom properties like Hook which are unique to mocks.

@nohwnd: Thoughts? I'm not very familiar with this mock-logic, but immediate (untested) thoughts would be to either:

• Add a test in Should-Invoke that throws an exception prior to calling Should-InvokeInternal if the resolved command is not a mock. That would hopefully force the user to always specify -ModuleName in the assertion for mocks created in a module-scope.
• Or maybe update Get-AssertMockTable so it doesn't return callhistory with module-scoped invocations when -ModuleName is not provided in Should -Invoke.

[JustinGrote]

I'll give it a shot and report back, I didn't know Should also had the module name scoping.

[JustinGrote]

@fflaten that worked like a charm for me. Makes sense that if the mock is in the module scope it would be tracking the invocations in the same scope.

Should -Invoke 'Remove-GithubRelease' -ModuleName Press -Exactly -Times $RemoveCount

[Glober777]

@fflaten, works great! Thanks a ton!

I wish it was more obvious when reading the error message.

[nohwnd]

@fflaten I think 1) needs to be done because it is not change to the current Mock behavior. We can suggest using the -ModuleName in the error, and even in log. But I am also not sure why we are resolving the command again, instead of attaching info to the history and using it. (If I got what is happening from your description correctly).