Pester 4 now fails on strings which match (No issue with Pester 3.4)

[nzspambot]

Hi, working on this pull: dsccommunity/WebAdministrationDsc#281

noticed with appveyor now running Pester 4 it is showing weird failures such as:

      [-] Should return the correct verbose message 122ms
        Expected: {Handler with name 'StaticFile' already exist}
        But was:  {Handler with name 'StaticFile' already exist}
        at line: 81 in C:\projects\xwebadministration\Tests\Unit\MSFT_xIISHandler.tests.ps1
        81:                     $result[0] | Should Be ($LocalizedData.HandlerExists -f $Name)
      [+] Should return False 8ms
    Context Handler is Present and Ensure = Absent
      [+] Should return False 103ms
      [+] Should not return a verbose message 14ms
    Context Handler is Present and Ensure = Present
      [-] Should return the correct verbose message 108ms
        Expected: {Handler with name 'StaticFile' is not present as requested}
        But was:  {Handler with name 'StaticFile' is not present as requested}
        at line: 110 in C:\projects\xwebadministration\Tests\Unit\MSFT_xIISHandler.tests.ps1
        110:                     $result[0] | Should Be ($LocalizedData.HandlerNotPresent -f $Name)
      [+] Should return False 7ms
  Describing MSFT_xIISHandler\Set-TargetResource
    Context Ensure = Present and Handler is NOT present
      [+] Should add the handler 167ms
      [-] Should call the right Verbose Message 11ms
        Expected: {Adding handler 'StaticFile'}
        But was:  {Adding handler 'StaticFile'}
        at line: 136 in C:\projects\xwebadministration\Tests\Unit\MSFT_xIISHandler.tests.ps1
        136:                     $message | Should Be ($LocalizedData.AddingHandler -f $mockName)
    Context Ensure = Absent and Handler IS present
      [+] Should add the handler 137ms
      [-] Should call the right Verbose Message 12ms
        Expected: {Removing handler 'StaticFile'}
        But was:  {Removing handler 'StaticFile'}
        at line: 153 in C:\projects\xwebadministration\Tests\Unit\MSFT_xIISHandler.tests.ps1
        153:                     $message | Should Be ($LocalizedData.RemovingHandler -f $mockName)

For some reason Pester thinks the Should Be is not correct. This is also the case for [String]False which fails vs [bool]$false which works

    [-] Changing Default virtualDirectoryDefaults 531ms
      Expected: {false}
      But was:  {False}
      at line: 60 in C:\projects\xwebadministration\Tests\Integration\MSFT_xWebsiteDefaults.Integration.Tests.ps1
      60:             $changedValue | should be $env:PesterVirtualDirectoryDefaults
There is no operation running currently. Stop will return without any action.

Full test output is here: https://ci.appveyor.com/project/PowerShell/xwebadministration/build/1.16.900.0

Full Tests are here: https://github.com/nzspambot/xWebAdministration/tree/xIISRemoteManagement/Tests

Also note the failures are outside of my PR i.e I haven't touched them

thanks

[nohwnd]

@nzspambot thanks for reporting back, I can reproduce the same problems when my actual value is scriptblock and not a string. I can also reproduce the problem with False when I put string as the expected value and boolean as the input value. The Should changed internally, and those are possibly some unforseen cases, or bugs on your or our side :). Could you please check what types of input you are providing to Expected and Actual in the "Should return the correct verbose message" and in "Return False when AuthenticationInfo is not correct -- should return false"?

The version 4 is still RC and we can do something about the way Should works and the messages describing why it failed could definitely be better.

Simple repro:

describe "a" { 
    it "b" { 
        {hello} | should be "hello"
    }
}

# [-] b 29ms
#   Expected: {hello}
#   But was:  {hello}
#   at line: 3 in
#   3:         {hello} | should be "hello"

describe "a" { 
    it "b" { 
        $false | should be False
    }
}

# [-] b 27ms
#   Expected: {False}
#   But was:  {False}
#   at line: 3 in
#   3:         $false | should be False

[nohwnd]

@nzspambot For more infomation about the possible cause please see #703.

[nohwnd]

Can you possibly run your tests against this version https://github.com/nohwnd/Pester/tree/FixShouldBe ?

[rkeithhill]

I'm seeing the same failure with this script:

$repoRoot = (Resolve-Path $PSScriptRoot\..).Path
$gitDir = Join-Path $repoRoot .git
Set-Location $gitDir -PassThru | Should BeExactly $gitDir

[nohwnd]

@rkeithhill same issue, same fix, the $expected needs to move to the left side of the -eq operator. Could you please you run the Pester version from the linked branch please? It should fix the issue, and not introduce any other issues. :)

$actual = Set-Location -PassThru
$expected = "$pwd"
$actual -eq $expected # is False
$expected -eq $actual # is True

[rkeithhill]

@nohwnd Yes, that FixShouldBe branch fixes it. Thanks!

[nzspambot]

@nohwnd LGTM thanks for that; will close this now