negative test filter requires at least one property (trait / category) to exist within the tested assembly

[dmccaffery]

Description

When executing the vstest runner with a negative filter on an assembly that does not contain at least one property of the specified type, an exception occurs.

Steps to reproduce

1. Create a new unit test project that does not define a category property or trait.
2. Execute vstest with a TestCaseFilter that has a negative check for the category property or trait.
3. Exception will occur indicating that no test contains the specified category property or trait.

Example:

dotnet test "<assembly>" --no-build --configuration "Debug" --filter "Category!=Platform-Not-MacOS"

(this will call vstest.console underneath with a --TestCaseFilter and --TestAdapterPath)

Expected behavior

All tests that do not have a category property or trait should execute based on the following logic: Category!=SomeValue is true when Category is not defined

Actual behavior

Exception filtering tests: No tests matched the filter because it contains one or more properties that are not valid (Category). Specify filter expression containing valid properties (DisplayName, FullyQualifiedName) and try again.

Environment

~ dotnet --info
.NET Command Line Tools (1.0.4)

Product Information:
 Version:            1.0.4
 Commit SHA-1 hash:  af1e6684fd

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.12
 OS Platform: Darwin
 RID:         osx.10.12-x64
 Base Path:   /Users/deavon/.dotnet/sdk/1.0.4

~ dotnet vstest
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

[Faizan2304]

The issue is with xunit. I opened a bug in xunit repo (xunit/xunit#1314) to track this. I am closing it here.

[Faizan2304]

Fixed!!

@jamescrosswell

[jrr]

I still have this problem, with the following environment:

• VS2017
• SDK-style project targeting net462
• xunit 2.3.1
• Microsoft.NET.Test.Sdk 15.5

Both dotnet test (2.0.0) and vstest.console.exe (15.0.26929.2) produce this output:

[xUnit.net 00:00:01.3961216] Foo.Test: Exception filtering tests: No tests matched the filter because it contains one or more properties that are not valid (Bug). Specify filter expression containing valid properties (DisplayName, FullyQualifiedName) and try again.

What did I miss?