0.15.7 throws ArgumentException analyzing code causing build to fail with AD0001 error

[martincostello]

From open-telemetry/opentelemetry-dotnet-contrib#3428, the CI fails (due to treating warnings as errors) to build some of our jobs with errors such as this:

CSC : error AD0001: Analyzer 'BenchmarkDotNet.Analyzers.Attributes.ParamsAttributeAnalyzer' threw an exception of type 'System.ArgumentException' with message 'Inconsistent syntax tree features (Parameter 'trees')'. [D:\a\opentelemetry-dotnet-contrib\opentelemetry-dotnet-contrib\test\OpenTelemetry.Exporter.Geneva.Benchmarks\OpenTelemetry.Exporter.Geneva.Benchmarks.csproj::TargetFramework=net10.0]

You can reproduce the issue by:

1. Cloning the repository at open-telemetry/opentelemetry-dotnet-contrib@3f30a36
2. cd .\test\OpenTelemetry.Exporter.Geneva.Benchmarks\
3. dotnet build

It's not clear from the error message which specific benchmark is causing the error.

[timcassell]

@silkfire PTAL

[m0sa]

Just dropped by to +1 this. Not sure if relevant, but the project where I can see this is using the new 10.0 SDK (10.0.100), but still targeting the net8.0 tfm.

[silkfire]

I'll take a look at it tonight. I wonder if it has to do with the syntax parser we use to check if a type is assignable to a field/variable and the C# version of the user's compilation differs from the one used by the parser.

[pvlst]

Having a similar issue. My project uses latest .NET 9.0.11 and the C# language version is set to 12.0, if that's somehow relevant.

CSC : warning AD0001: Analyzer 'BenchmarkDotNet.Analyzers.Attributes.ParamsAttributeAnalyzer' threw an exception of type 'System.ArgumentException' with message 'Inconsistent language versions (Parameter 'syntaxTrees')'.

[AndreyAkinshin]

@adamsitnik @timcassell @silkfire Since 0.15.7 causes such troubles, I unlisted it from nuget.org to prevent more user complaints. This version will not be listed in the search results, so users will hopefully not try to upgrade.

[silkfire]

It was as I suspected, it had to do with the language version of the CSharpSyntaxTree.ParseText method, it has to match the version used in the compilation.

The reason it never manifested in the unit tests is because both the analyzer verifier (test) and the base analyzer default to the latest C# version, so they will always match.

I've fixed the error locally and added a suitable test. To make it support projects that target .NET 10 (C# 14), I had to upgrade Microsoft.CodeAnalysis.CSharp to v5.x which is a pre-release version of Roslyn.

Unfortunately I can't get it to work with .NET 10. I printed a custom exception message to compare the language versions, and despite that they're equal, it throws the Inconsistent language versions error. No idea why.

warning AD0001: Analyzer 'BenchmarkDotNet.Analyzers.Attributes.ParamsAttributeAnalyzer' threw an exception of type 'System.Exception' with message 'Compilation version: CSharp14; syntax tree parse options language version: CSharp14

This is the method used to determine that all syntax trees share the same language version (in the class CSharpCompilation):

private static LanguageVersion CommonLanguageVersion(ImmutableArray<SyntaxTree> syntaxTrees)
{
    LanguageVersion? result = null;
    foreach (var tree in syntaxTrees)
    {
        var version = ((CSharpParseOptions)tree.Options).LanguageVersion;
        if (result == null)
        {
            result = version;
        }
        else if (result != version)
        {
            throw new ArgumentException(CodeAnalysisResources.InconsistentLanguageVersions, nameof(syntaxTrees));
        }
    }

    return result ?? LanguageVersion.Default.MapSpecifiedToEffectiveVersion();
}

[timcassell]

So... do we need help from someone on the Roslyn team? You could open an issue in dotnet/roslyn and link this issue.

[silkfire]

I found a way to debug the analyzer while it's being run by a project that references BenchmarkDotNet and also peek into decompiled 3rd party code (Microsoft.CodeAnalysis), so I'll see if I can find something that way.

It seems to be happening here:

It's comparing the syntax trees of two source code files by their "features", and some kind of interceptor is diffing so it's throwing.
It seems to be an AssemblyInfo file.

[prezaei]

Same issue here

[silkfire]

I figured it out, I'll post a PR tomorrow; we can just exclude all syntax trees because the only one relevant is the dummy one that contains the code wherein we're trying to test whether a field or variable is assignable to a particular type.

[silkfire]

This took a bit longer than I thought as I discovered further issues with the convertible check for enums.

If the type isn't fully qualified, then the dummy syntax tree compilation won't know how to resolve it, leading to false positive like Unexpected type for argument value 'MyEnum.Value'. Expected 'MyNamespace.MyEnum' but found 'MyNamespace.MyEnum'.

I needed to find a way to add the namespace using and add suitable tests. Almost done.

[silkfire]

@timcassell PR is up. This took way longer than I anticipated! This interaction between (implicit) conversion, compilation and syntax trees is a rabbit hole that can be really hard to get out from. Hopefully should be more stable now.