Pester test Context failures not showing up in Problems window

[tillig]

Issue Description

If you have a Pester test where a BeforeAll or AfterAll block fails that failure doesn't show up in the Problems window. Test failures do properly display.

Here's a repro test you can use to try it out:

Describe "Reproduce Issue" {
    Context "Failing BeforeAll in Context block" {
        BeforeAll {
            # In a "real" scenario, this is
            # $pathToNuGet = Get-Command nuget.exe | Select-Object -ExpandProperty Source
            # &mono $pathToNuGet do-some-things-with-nuget.exe
            # to execute NuGet on a Linux/OS X box. The failure I'm seeing in the context
            # happens when mono isn't available, and possibly nuget.exe is also not there.
            $pathToCommand = Get-Command nosuchcommand.exe | Select-Object -ExpandProperty Source
            &othermissingcommand $pathToCommand
        }
    }
    It "has two failures - context and test" {
        $False | Should -Be $True
    }
}

The output of this in the console looks like:

> Executing task: Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true} <

    ____            __
   / __ \___  _____/ /____  _____
  / /_/ / _ \/ ___/ __/ _ \/ ___/
 / ____/  __(__  ) /_/  __/ /
/_/    \___/____/\__/\___/_/
Pester v4.9.0
Executing all tests in '.'

Executing script /Users/tillig/dev/repro/Repro.Tests.ps1

  Describing Reproduce Issue

    Context Failing BeforeAll in Context block
Get-Command: /Users/tillig/dev/repro/Repro.Tests.ps1:9:30
Line |
   9 |              $pathToCommand = Get-Command nosuchcommand.exe | Select-O …
     |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The term 'nosuchcommand.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
     | spelling of the name, or if a path was included, verify that the path is correct and try again.

      [-] Error occurred in Context block 0ms
        At /Users/tillig/dev/repro/Repro.Tests.ps1:10 char:14
        The term 'othermissingcommand' is not recognized as the name of a cmdlet, function, script file, or operable program.
        Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
    [-] has two failures - context and test 77ms
      at <ScriptBlock>, /Users/tillig/dev/repro/Repro.Tests.ps1: line 14
      14:         $False | Should -Be $True
      Expected $true, but got $false.
Tests completed in 508ms
Tests Passed: 0, Failed: 2, Skipped: 0, Pending: 0, Inconclusive: 0 

Terminal will be reused by tasks, press any key to close it.

The Problems window does display the test failure - 'has two failures - context and test' but it does NOT display the failure in the test context itself.

If you switch the test above so it's $True | Should -Be $True (so there's an assertion and it passes), only the Context will fail. Re-run the test and you'll see there's nothing in the Problem window.

Attached Logs

I didn't notice anything in the logs that indicated a failure around this, it appears it's a problem matcher issue. I can gather logs if you think it'd help.

Environment Information

Visual Studio Code

Name	Version
Operating System	Darwin x64 19.2.0
VSCode	1.41.1
PowerShell Extension Version	2020.1.0

PowerShell Information

Name	Value
PSVersion	7.0.0-rc.2
PSEdition	Core
GitCommitId	7.0.0-rc.2
OS	Darwin 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST 2019, root:xnu-6153.61.1~20/RELEASE_X86_64
Platform	Unix
PSCompatibleVersions	1.0 2.0 3.0 4.0 5.0 5.1.10032.0 6.0.0 6.1.0 6.2.0 7.0.0-rc.2
PSRemotingProtocolVersion	2.3
SerializationVersion	1.1.0.1
WSManStackVersion	3.0

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)

Extension	Author	Version
auto-close-tag	formulahendry	0.5.6
auto-rename-tag	formulahendry	0.1.1
azure-account	ms-vscode	0.8.8
azurerm-vscode-tools	msazurermtools	0.8.3
beautify	HookyQR	1.5.0
code-runner	formulahendry	0.9.15
code-settings-sync	Shan	3.4.3
color-info	bierner	0.5.1
cpptools	ms-vscode	0.26.3
csharp	ms-vscode	1.21.9
debugger-for-chrome	msjsdiag	4.12.6
dotenv	mikestead	1.0.1
dotnet-test-explorer	formulahendry	0.7.1
ecdc	mitchdenny	1.3.0
EditorConfig	EditorConfig	0.14.4
gitlens	eamodio	10.2.0
Go	ms-vscode	0.12.0
guides	spywhere	0.9.3
html-css-class-completion	Zignd	1.19.0
java	redhat	0.55.1
markdown-preview-enhanced	shd101wyy	0.5.1
material-theme	zhuangtongfa	3.2.1
msbuild-project-tools	tintoy	0.3.3
mssql	ms-mssql	1.8.0
nc-gcode	ML	0.13.0
ng-template	Angular	0.900.6
npm-intellisense	christian-kohler	1.3.0
output-colorizer	IBM	0.1.2
path-intellisense	christian-kohler	1.4.2
polacode	pnp	0.3.4
powershell	ms-vscode	2020.1.0
python	ms-python	2020.1.58038
remote-wsl	ms-vscode-remote	0.41.7
restructuredtext	lextudio	117.0.0
rewrap	stkb	1.9.1
sort-lines	Tyriar	1.9.0
sublime-keybindings	ms-vscode	4.0.6
terraform	mauve	1.4.0
todo-tree	Gruntfuggly	0.0.166
vscode-aks-tools	ms-kubernetes-tools	0.0.4
vscode-arduino	vsciot-vscode	0.2.29
vscode-docker	ms-azuretools	0.9.0
vscode-eslint	dbaeumer	2.0.15
vscode-icons	vscode-icons-team	9.7.0
vscode-java-debug	vscjava	0.24.0
vscode-java-dependency	vscjava	0.8.0
vscode-java-pack	vscjava	0.8.1
vscode-java-test	vscjava	0.22.1
vscode-json-stable-stringify	TravisIllig	1.1.0
vscode-kubernetes-tools	ms-kubernetes-tools	1.0.9
vscode-markdownlint	DavidAnson	0.33.0
vscode-maven	vscjava	0.20.2
vscode-npm-script	eg2	0.3.10
vscode-typescript-tslint-plugin	ms-vscode	1.2.3
vscode-yaml	redhat	0.6.1
vscodeintellicode	VisualStudioExptTeam	1.2.4
xml	DotJoshJohnson	2.5.0

[SydneyhSmith]

@tillig thanks for reporting this issue, we are able to reproduce this. It is not an issue we will likely have the resources to solve in the short term (although we do welcome PRs and can point you in the right direction if you are interested)...it is an issue that may get addressed in the medium term as we will need to make changes to align with Pester 5...thanks!

[SydneyhSmith]

This is where the relevant code is:

vscode-powershell/package.json

Lines 344 to 364 in 50a2611

	"problemMatchers": [
	{
	"name": "pester",
	"owner": "powershell",
	"fileLocation": [
	"absolute"
	],
	"severity": "error",
	"pattern": [
	{
	"regexp": "^\\s*(?:\\[-\\]\\s+)(.*?)(?:\\s+\\d+\\.?\\d*\\s*m?s)\\s*$",
	"message": 1
	},
	{
	"regexp": "^\\s+at\\s+[^,]+,\\s*(.*?):\\s+line\\s+(\\d+)$",
	"file": 1,
	"line": 2
	}
	]
	}
	],

[tillig]

OK, so it sounds like a problem matcher thing to you, too? I can probably take a look at it.

[tillig]

Done. The issue was in the location (line/file) matching part. Submitted a PR for that.