AnalyzerTest.IsSubjectToExclusion constrains DiagnosticsID

[ladeak]

Visual Studio Community Version 16.6.0 C# Analyzer templates test package uses new packages (version 1.0.1-beta1.*):

Microsoft.CodeAnalysis.CSharp.Analyzer.Testing.MSTest
Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.MSTest
Microsoft.CodeAnalysis.CSharp.CodeRefactoring.Testing.MSTest

The test packages seem to require the Analyzer ID to start with "AD", otherwise AanyzerTests.VerifyDiagnosticsAsync seems to exclude the expected diagnostics:

            var expectedResults = expected
                .Where(x => !IsSubjectToExclusion(x, sources))
                .Select(x => IsInSourceFile(x, sources) ? x.WithLineOffset(1) : x)
                .ToArray();

Stacktrace:

Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest<Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>.VerifyGeneratedCodeDiagnosticsAsync(System.Collections.Immutable.ImmutableArray<Microsoft.CodeAnalysis.Diagnostics.DiagnosticAnalyzer> analyzers, (string filename, Microsoft.CodeAnalysis.Text.SourceText content)[] sources, (string filename, Microsoft.CodeAnalysis.Text.SourceText content)[] additionalFiles, Microsoft.CodeAnalysis.Testing.ProjectState[] additionalProjects, Microsoft.CodeAnalysis.MetadataReference[] additionalMetadataReferences, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expected, Microsoft.CodeAnalysis.Testing.IVerifier verifier, System.Threading.CancellationToken cancellationToken) Line 257	C#
Microsoft.CodeAnalysis.Analyzer.Testing.dll!Microsoft.CodeAnalysis.Testing.AnalyzerTest<Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>.VerifyDiagnosticsAsync((string filename, Microsoft.CodeAnalysis.Text.SourceText content)[] sources, (string filename, Microsoft.CodeAnalysis.Text.SourceText content)[] additionalFiles, Microsoft.CodeAnalysis.Testing.ProjectState[] additionalProjects, Microsoft.CodeAnalysis.MetadataReference[] additionalMetadataReferences, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expected, Microsoft.CodeAnalysis.Testing.IVerifier verifier, System.Threading.CancellationToken cancellationToken) Line 238	C#
Microsoft.CodeAnalysis.CodeFix.Testing.dll!Microsoft.CodeAnalysis.Testing.CodeFixTest<Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>.RunAsync(System.Threading.CancellationToken cancellationToken) Line 226	C#
Microsoft.CodeAnalysis.CodeFix.Testing.dll!Microsoft.CodeAnalysis.Testing.CodeFixVerifier<ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerAnalyzer, ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerCodeFixProvider, Microsoft.CodeAnalysis.CSharp.Testing.CSharpCodeFixTest<ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerAnalyzer, ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerCodeFixProvider, Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>, Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>.VerifyCodeFixAsync(string source, Microsoft.CodeAnalysis.Testing.DiagnosticResult[] expected, string fixedSource) Line 81	C#
Microsoft.CodeAnalysis.CodeFix.Testing.dll!Microsoft.CodeAnalysis.Testing.CodeFixVerifier<ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerAnalyzer, ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerCodeFixProvider, Microsoft.CodeAnalysis.CSharp.Testing.CSharpCodeFixTest<ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerAnalyzer, ConvertNxUnitAnalyzer.ConvertNxUnitAnalyzerCodeFixProvider, Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>, Microsoft.CodeAnalysis.Testing.Verifiers.MSTestVerifier>.VerifyCodeFixAsync(string source, Microsoft.CodeAnalysis.Testing.DiagnosticResult expected, string fixedSource) Line 61	C#

[sharwell]

The test packages seem to require the Analyzer ID to start with "AD", otherwise AanyzerTests.VerifyDiagnosticsAsync seems to exclude the expected diagnostics ...

This code is part of VerifyGeneratedCodeDiagnosticsAsync, where the following comment is automatically added to the top of the file:

// <auto-generated>

When this file is present, analyzer diagnostics are generally expected to not be reported because users will not be able to fix them. For additional details and the available approaches to resolve this, see #422 (comment).

[sharwell]

This was fixed by #569. If the analyzer calls ConfigureGeneratedCodeAnalysis, then the expected test result will adhere to that configuration without needing to specify TestBehaviors.SkipGeneratedCodeCheck.