[Poison] Add detection of reference assemblies

[crummel]

Currently, the leak detection infrastructure only poisons the previously-source-built nupkgs. We've talked with the runtime team and we also shouldn't ever be shipping reference assemblies - we can augment the poison reporting to detect these to help with this goal:

• MarkAndCatalog task should include the ref packages to record the hashes of those assemblies as well.
• CheckForPoison should include a new PoisonType, ReferenceAttribute or similar, that checks for the ReferenceAssembly attribute on shipped assemblies.

[ellahathaway]

In discussing this issue with @NikolaMilosavljevic, it seems that adding in this feature might require some refactoring of the poison steps. Currently, there are reference assemblies are coming from the SBRP repo which is built after we poison the files, and there are reference assemblies in the previously-source-built artifacts package.

We've never done poisoning steps during build, which addressing this issue would require us to do should we use the reference assemblies from the SBRP repo. On the other hand, there are also the reference assemblies coming from the PSB artifacts package. Do we know which ones are used during the build?

Should we decide to use the assemblies from the SBRP repo, this issue might warrant some brainstorming.

[ellahathaway]

In trying to address this, I'm having trouble finding some of the .dll files that are present in the SDK.

One reference assembly in particular is Microsoft.Win32.SystemEvents.dll. I expected this assembly to be in a nuget package or as a plain binary in the dotnet repo, but no such luck. The best I can trace it to is ./installer/src/SourceBuild/content/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Debug/net8.0/.dotnet/sdk/9.0.100-alpha.1.23431.1/Microsoft.Win32.SystemEvents.dll and ./dotnet/src/installer/artifacts/source-build/self/src/artifacts/bin/redist/Release/dotnet/sdk/9.0.100-alpha.1.23422.1/Microsoft.Win32.SystemEvents.dll. Is there an explanation for why this assembly is not present in PSB or SBRP?

[mthalman]

It is in SBRP: https://github.com/dotnet/source-build-reference-packages/tree/main/src/referencePackages/src/microsoft.win32.systemevents/7.0.0. It would be in /vmr/prereqs/packages/reference during the build.

[ellahathaway]

I should've been more specific. When poisoning SBRP (which included /vmr/prereqs/packages/reference and /vmr/artifacts/obj/x64/Release/blob-feed/packages/SourceBuildReferencePackages, done in different builds), I found that the reference assembly would be in the poison catalog, but the assembly was still in the resulting sdk and was not in the poison usage report. This led me to realize that the sha256 hashes of the ref assemblies in the catalog, all with the same name as the assembly in the sdk, did not match the hash of the assembly in the sdk. Additionally, the assembly in the sdk was missing the poison marker, indicating that it had not been poisoned and cataloged.

I've found multiple instances of the reference assembly with the same name in the repo, but none have the same sha256 hash and binary code as the reference assembly in the sdk. Here is a report of the sha256 hash of the reference assembly in the sdk and the other hashes of matching names in /vmr/prereqs/

Tldr: The reference assembly is not being identified as a leak when I poison SBRP, leading me to wonder where this reference assembly is being pulled from if it's not in SBRP.

[ellahathaway]

Most recent update:

https://github.com/dotnet/installer/pull/17339/files#r1332210220

[ellahathaway]

Updated poison reports after adding SBRP attribute to source-build-reference-packages.

• preview7: https://gist.github.com/ellahathaway/158aedc428ed1cf27837e91b030053c7#file-dotnet-dotnet-preview7-poison-leaks
• release/8.0.1xx: https://gist.github.com/ellahathaway/c51939c465d666c3d1609a05c10422ea#file-dotnet-dotnet-release-8-0-1xx-poison-report

[ellahathaway]

Closing with merge of dotnet/installer#17339