Prevent unintended configuration changes to .NET (Roslyn) analyzers.
There are many ways to configure diagnostic warning and error levels in a .NET build, and understanding how they all interact can be tricky to get correct. .NET / MSBuild support all these mechanisms (and probably more!) to configure what analyzers are enabled and at what severity level:
- Analyzers provided in the SDK (docs)
- Analyzer NuGet packages (example)
.editorconfig(docs).globalconfig(docs).ruleset(docs)WarningLevel(docs)AnalysisLevel(docs)TreatWarningsAsErrors(docs)WarningsAsErrors(docs)WarningsNotAsErrors(docs)NoWarn(docs)
That's a lot to keep track of!
With SquiggleCop, any change to project files or build scripts produces an easy to understand (and diff!) baseline file that shows the consequences of the change:
| Code Change |
|---|
--- a/sample.csproj
+++ b/sample.csproj
@@ -3,7 +3,7 @@
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
- <AnalysisLevel>5</AnalysisLevel>
+ <AnalysisLevel>6</AnalysisLevel>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable> |
| Baseline File Diff |
--- a/SquiggleCop.Baseline.yaml
+++ b/SquiggleCop.Baseline.yaml
@@ -57,8 +57,8 @@
- {Id: CA1401, Title: P/Invokes should not be visible, Category: Interoperability, DefaultSeverity: Note, IsEnabledByDefault: true, EffectiveSeverities: [Note], IsEverSuppressed: false}
- {Id: CA1416, Title: Validate platform compatibility, Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [Warning], IsEverSuppressed: false}
- {Id: CA1417, Title: Do not use 'OutAttribute' on string parameters for P/Invokes, Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [Warning], IsEverSuppressed: false}
-- {Id: CA1418, Title: Use valid platform string, Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [None], IsEverSuppressed: true}
-- {Id: CA1419, Title: Provide a parameterless constructor that is as visible as the containing type for concrete types derived from 'System.Runtime.InteropServices.SafeHandle', Category: Interoperability, DefaultSeverity: Note, IsEnabledByDefault: true, EffectiveSeverities: [None], IsEverSuppressed: true}
+- {Id: CA1418, Title: Use valid platform string, Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [Warning], IsEverSuppressed: false}
+- {Id: CA1419, Title: Provide a parameterless constructor that is as visible as the containing type for concrete types derived from 'System.Runtime.InteropServices.SafeHandle', Category: Interoperability, DefaultSeverity: Note, IsEnabledByDefault: true, EffectiveSeverities: [Note], IsEverSuppressed: false}
- {Id: CA1420, Title: 'Property, type, or attribute requires runtime marshalling', Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [None], IsEverSuppressed: true}
- {Id: CA1421, Title: This method uses runtime marshalling even when the 'DisableRuntimeMarshallingAttribute' is applied, Category: Interoperability, DefaultSeverity: Note, IsEnabledByDefault: true, EffectiveSeverities: [None], IsEverSuppressed: true}
- {Id: CA1422, Title: Validate platform compatibility, Category: Interoperability, DefaultSeverity: Warning, IsEnabledByDefault: true, EffectiveSeverities: [None], IsEverSuppressed: false} |
SquiggleCop parses compiler output to create a baseline file of all .NET (Roslyn) analyzer rules and their configured severity levels. This baseline file should be checked into source control. On subsequent runs the new baseline is compared to the existing file. If baselines don't match, either an unexpected configuration change was made and should be fixed, or the baseline should be updated to document the new, expected configuration.
SquiggleCop is available in both MSBuild task and CLI tool form to make it easy to integrate into your existing development processes.
SquiggleCop uses SARIF v2.1 files to work its magic. SquiggleCop automatically
enables SARIF logs if needed and places them in the obj/ folder. If you want / need to customize the SARIF output
path, see Set the SARIF output path.
The CLI tool is designed for ad-hoc validation and interacting with baseline files. Install the CLI tool by running this command:
dotnet tool install SquiggleCop.ToolTo generate or diff a new baseline, run the generate command like this:
Usage: dotnet squigglecop generate [--auto-baseline] [--output <String>] [--context <Int32>] [--help] sarif
Arguments:
0: sarif The SARIF log to generate a baseline for (Required)
Options:
-a, --auto-baseline Automatically update baseline if necessary
-o, --output <String> The output path for the baseline file
-c, --context <Int32> Number of context lines to use in the diff (Default: 3)
-h, --help Show help message
The Tasks package automatically integrates SquiggleCop into the build using MSBuild. This package is designed for generating baselines as part of a large build and to continuously validate baselines and prevent unintentional build changes.
Add the Tasks package like this:
dotnet add package SquiggleCop.TasksIf you use Central Package Management (CPM), you can use a
GlobalPackageReference
to add SquiggleCop to every project automatically.
<Project>
<ItemGroup>
<GlobalPackageReference Include="SquiggleCop.Tasks" Version="{{version}}" />
</ItemGroup>
</Project>If a new baseline doesn't match the existing file, SquiggleCop emits MSBuild warning SQ2000: Baseline mismatch.
Either use the SquiggleCop CLI to create a new baseline, or enable automatic baselining by setting:
<Project>
<PropertyGroup>
<SquiggleCop_AutoBaseline>true</SquiggleCop_AutoBaseline>
</PropertyGroup>
</Project>If autobaseline is on, be sure to review any changes to the baseline file before committing your code.
Caution
If you turn auto-baseline on, be sure to turn it off in CI. Otherwise SquiggleCop may not be able to warn about potential issues!
A baseline file is a YAML file with a repeating structure. Here's a single rule entry:
- Id: CA1000
Title: Do not declare static members on generic types
Category: Design
DefaultSeverity: Note
IsEnabledByDefault: true
EffectiveSeverities: [Note, None]
IsEverSuppressed: trueThis is the ID of the diagnostic.
The title of the diagnostic. Note that some diagnostics have multiple titles for a single ID. For instance, depending
on how it's configured, the title of IDE0053 can be either "Use block body for lambda expression" or "Use expression
body for lambda expression".
The category of the diagnostic. See Analyzer Configuration for more information.
The diagnostic's default severity.
If the diagnostic is enabled by default.
The severity or severities of a diagnostic once all options have been considered (i.e. rulesets, .editorconfig,
.globalconfig, etc.). One common way to end up with multiple effective severities is to use different .editorconfig files
for different parts of the codebase. Note that inline suppressions will not show up here, instead they show up in
IsEverSuppressed.
true if the diagnostic is ever suppressed at a call site. Common ways to do this are:
#pragma[SuppressMessage]<AnalysisLevel>
The easiest way to debug a baseline mismatch that occurs in CI but doesn't occur locally is to:
- Upload the SARIF files from the build
- Use the SquiggleCop CLI to generate a new baseline from the CI SARIF file and compare it to the checked in baseline
Upload your SARIF reports as pipeline artifacts to help narrow down issues.
- name: Upload SARIF logs
uses: actions/upload-artifact@v4
if: success() || failure() # Upload logs even if the build failed
with:
name: SARIF logs
path: ./artifacts/**/*.sarif # Modify as necessary to point to your sarif files- task: CopyFiles@2
displayName: 'Copy SARIF files to Artifact Staging'
condition: succeededOrFailed() # Upload logs even if the build failed
inputs:
contents: 'artifacts\**\*.sarif' # Modify as necessary to point to your sarif files
targetFolder: '$(Build.ArtifactStagingDirectory)\sarif'
cleanTargetFolder: true
overWrite: true
- task: PublishPipelineArtifact@1
displayName: 'Publish SARIF files as Artifacts'
condition: succeededOrFailed() # Upload logs even if the build failed
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)\sarif'
publishLocation: 'pipeline'
artifact: 'sarif'If the ErrorLog
property is unset, SquiggleCop sets it just before the CoreCompile target. To set a custom SARIF output path, set the
property to something like this:
<ErrorLog>$(MSBuildProjectFile).sarif,version=2.1</ErrorLog>Tip
We recommend you add *.sarif to your .gitignore file
or set the property on the command-line as part of ad-hoc validation:
dotnet build -p:ErrorLog=diagnostics.sarif%2cversion=2.1Important
The comma or semi-colon character in the log path must be XML-escaped
By default, SquiggleCop expects the baseline file to be named SquiggleCop.Baseline.yaml and placed next to the project
file. To specify a custom path to the baseline file, add an item to AdditionalFiles that points to the baseline file:
<Project>
<ItemGroup>
<AdditionalFiles Include="/path/to/SquiggleCop.Baseline.yaml" />
</ItemGroup>
</Project>Often, projects use TreatWarningsAsErrors in CI builds to prevent warnings from entering the main branch.
However, toggling TreatWarningsAsErrors also changes the effective severity of analyzer diagnostics, which can lead to
unnecessary churn in baseline files. If your project or development workflow toggles TreatWarningsAsErrors between CI
and local development, also toggle the SquiggleCop_Enabled property based on the same logic.
Here's a sample that toggles TreatWarningsAsErrors based on the
ContinuousIntegrationBuild
property:
<Project>
<PropertyGroup>
<PedanticMode Condition=" '$(PedanticMode)' == '' ">$([MSBuild]::ValueOrDefault('$(ContinuousIntegrationBuild)', 'false'))</PedanticMode>
<TreatWarningsAsErrors>$(PedanticMode)</TreatWarningsAsErrors>
<SquiggleCop_Enabled>$(PedanticMode)</SquiggleCop_Enabled>
</PropertyGroup>
</Project>Baseline files are written in UTF-8 encoding without a BOM. Baseline files use the \n line ending on all
platforms. SquiggleCop's own diffing algorithm ignores end of line differences to avoid unnecessary issues, however
depending on your .gitattributes settings line endings may be normalized to other values. If Git's line ending
normalization is causing issues, consider setting the following in your .gitattributes file:
# Store SquiggleCop baselines as lf regardless of platform
SquiggleCop.Baseline.yaml text eol=lf
And then run git add --renormalize . to update Git with the re-normalized files.
Icon 'fractal' by Bohdan Burmich from Noun Project (CC BY 3.0)
This means that somehow you're building differently locally than you are in CI. Common causes are:
- Different MSBuild parameters locally vs CI
- Also check if settings are based off the
$(ContinuousIntegrationBuild)property, which some CI providers set
- Also check if settings are based off the
- Different SDK versions
- Use a global.json to set the same SDK version locally and in CI
- New SDK feature versions can introduce new analyzers so we suggest limiting
rollForwardto patch updates, or disable entirely
This is probably because you've set rules in an .editorconfig, so it only applies to files that match the section. Importantly,
this is true even for the root .editorconfig, as projects can contain files from outside the repo / project root, and thus the
compiler is correctly (albeit pedantically) reporting that the project could have files where .editorconfig rules don't apply.
If you want modify an analyzer rule project-wide, use a .globalconfig file.