Source generator dependency unable to be resolved #52017
Comments
dotnet-issue-labeler
bot
added
Area-Compilers
untriaged
|
@Turnerj you must package all dependencies into your nuget package. The full set up dependencies you need to deploy are:
|
|
Thanks for the information @jmarolf - it sucks but is good to know why this is happening. Is there any plans to changing this behaviour for dependencies used inside source generators? While source generators allow a lot of flexibility, some advanced functionality (like what we do in Schema.NET) is difficult to implement without having dependencies. |
|
Edit: Ignore below, I re-read the actual warning message and missed the " or one of its dependencies" which would explain the message specifically. Actually I have an additional question, while what you mentioned makes sense, why would the error message be specifically that it couldn't find Like with what you described, I would expect to see an warning saying it couldn't find one of those dependencies. Saying that it couldn't find |
There isn't a restriction here on having dependencies just a requirement that source generators deploy their dependencies with them.
That minor release added a lot more dependencies and that is reflecting in this issue you are hitting.
This is a case where we are reflecting the error message that the .NET runtime provides when it attempts to load your DLL. Essentially we ask the runtime to load your generator and the runtime is providing the exception and the diagnostic here about the missing dependencies. It's not an error that we are directly providing. Closing as this isn't an issue in the compiler. Happy to continue chatting here to help you work through this though and make sure your deployment is successful. |
jaredpar
added
Feature - Source Generators
I guess my main issue here is that where one would normally just install the specific dependency they need, for a source generator I need to specify the entire tree of dependencies in my csproj file manually. While I'm sure a mountain of work has gone into making source generators work, this (and debugging source generators) make it feel somewhat incomplete. While @jmarolf went out of their way to list all of the dependencies in my case (thank you!), basically I'm wondering why some combination of NuGet restore/compilation process isn't doing this automatically. NuGet restore already has to work out all the dependencies anyway however this current process of manually adding everything is quite intensive, especially for when package updates happen as I would need to re-check to see if any new dependencies are added. So would there be any plans on changing this behaviour and having the compiler (or some other part of the build process) automatically handle this? |
|
I've been looking into ways that a user can avoid manually specifying transient dependencies for a source generator and have something working with the following: <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup Label="Package References">
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="3.9.0" />
<PackageReference Include="System.Text.Json" Version="5.0.1" PrivateAssets="all" />
</ItemGroup>
<PropertyGroup>
<GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>
<Target Name="GetDependencyTargetPaths" AfterTargets="ResolvePackageDependenciesForBuild">
<ItemGroup>
<TargetPathWithTargetPlatformMoniker Include="@(ResolvedCompileFileDefinitions)" IncludeRuntimeDependency="false" />
</ItemGroup>
</Target>
</Project>This effectively adds: This isn't perfect as it adds a few other libraries against My question though goes back to: Why isn't something like this done automatically for projects with source generators anyway? Honestly, I don't know really if this is much a Roslyn issue now or a default targets issue with MSBuild or .NET itself. I want to push for having this automatically handled by some part of the build process so users don't need to bother with manually configuring it. Any advice on where this would be best to further discuss would be appreciated! |
|
@jaredpar can you comment on that? |
|
What @Turnerj recommended did not work for me: This is a source generator used against a .net 5 project. <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<LangVersion>preview</LangVersion>
<BuildOutputTargetFolder>analyzers</BuildOutputTargetFolder>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.2" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="3.9.0" />
<PackageReference Include="Humanizer.Core" Version="2.10.1" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="5.0.0" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Buffers" Version="4.5.1" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Memory" Version="4.5.4" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Numerics.Vectors" Version="4.5.0" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="5.0.0" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Text.Encodings.Web" Version="5.0.1" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Threading.Tasks.Extensions" Version="4.5.4" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Text.Json" Version="5.0.2" GeneratePathProperty="true" PrivateAssets="all" />
</ItemGroup>
<PropertyGroup>
<GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>
<Target Name="GetDependencyTargetPaths">
<ItemGroup>
<TargetPathWithTargetPlatformMoniker Include="$(PKGHumanizer_Core)\lib\netstandard2.0\Humanizer.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Text_Json)\lib\netstandard2.0\System.Text.Json.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Text_Encodings_Web)\lib\netstandard2.0\System.Text.Encodings.Web.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Buffers)\lib\netstandard2.0\System.Buffers.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Memory)\lib\netstandard2.0\System.Memory.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Numerics_Vectors)\lib\netstandard2.0\System.Numerics.Vectors.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Runtime_CompilerServices_Unsafe)\lib\netstandard2.0\System.Runtime.CompilerServices.Unsafe.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Threading_Tasks_Extensions)\lib\netstandard2.0\System.Threading.Tasks.Extensions.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGMicrosoft_Bcl_AsyncInterfaces)\lib\netstandard2.0\Microsoft.Bcl.AsyncInterfaces.dll" IncludeRuntimeDependency="false" />
</ItemGroup>
</Target>
</Project> |
|
@kamronbatman I've found recently my solution has some issues - I don't think I've got the right combination of I may look at other solutions to automate this though for now, I'll put up with the occasional error during builds. I still believe the compiler should be doing this for us - tempted to raise it as another issue to at least get the ball rolling as it seems the compiler team isn't looking at this thread anymore. |
|
I literally can't get this to work no matter what I do. I'll have to switch to another Json package and lick my wounds. This is not worth my time and I wouldn't expect anyone to be an expert in the inner workings of msbuild/csproj. |
Do you have the binlog where this error was produced? Based on the error my expectation would be that you didn't pass this dependency via
I'm sorry you're having difficulty here, we realize this is a rough area. Unfortunately at the moment but unfortunately compilation execution is not an easy story right now. The compiler executes on a variety of runtimes and frameworks right now and analyzers / generators have to support all of them. Dependencies are challenging to reconcile easily with these restrictions and what does / does not need to be packaged is not easily automatable. In the future when the compiler moves to .NET Core only this will get easier but there is still a bit of time before that happens. |
|
@jaredpar I don't pass anything via As for the binlog, I'll generate that later today. To add to the frustration, I thought we were done with old school msbuild style passing of dependencies with the newest csproj spec. 😆 Note for anyone having this issue too: |
All DLLs in the same folder as the analyzer will essentially be passed via This is why I was asking for the binlog though. It will make it apparent what is happening here. |
|
Hey @jaredpar - could I get your feedback on my "trick" earlier in the thread? That is, using More importantly than that for me though, is some sort of automated behaviour by the compiler for adding transient dependencies to |
|
That level of MSBuild magic is a bit beyond me. @chsienki can likely better say. |
The
As you said above however, the trouble with this approach is it doesn't discriminate and basically adds _everything _ as an analyzer dependency. For small projects or things under your control that might be ok, but will slow the compilation as the list gets bigger, and worse potentially lead to really hard to debug version mismatches when you start adding more dependencies.
Its definitely on our radar, and something that has come up more since the introduction of source generators. The problem existed with analyzers too, but authors were just less likely to use dependencies. We've been discussing a range of steps that would improve all these scenarios. Most of that work is going to be outside of Roslyn (though we'll obviously help) so you're probably best opening an issue in http://github.com/dotnet/sdk to get the ball rolling. |
|
Hmm, this has got me thinking though. @jmarolf it looks like Can we just hook that task to return the dependencies as part of the analyzers directly (or even better, as an |
|
Thanks for the information @chsienki ! I've raised an issue on the SDK repo here: dotnet/sdk#17775 |
|
I am seeing this when trying to build a project that references my generator via
I have the generator pack that dependency up in its nupkg, but this isn't using the package to run the generator. How do I get this dependency available for the generator when it is used via ETA: I also have the dependencies written to the generator project's output directory: |
|
Using the clue of:
I changed the project that is referencing the generator project to reference the depended upon assembly as an analyzer, and that made the dependency work. I have tried to work down the dependency graph from this, but have had no luck in getting past this message in Visual Studio:
And
Does P2P just not support the generator having any dependencies, Even other Microsoft.CodeAnalysis.* assemblies? I would really like my generated code to be formatted. |
|
trying to use System.Text.Json in a source generator is failing for me also ... going to start trying the stuff in this thread to see if I can get past it ... and things were going so well with source generators ... |
I was able to get it to work using |
|
woohoo ! thanks ! at first I switched to to 5.0.0 and was still getting the same exception when calling BUT ... then I decided to try adding this incantation to the csproj and BOOM it worked. Now, should I spend my day learning about the above stuff (TargetPathWithTargetPlatformMoniker/etc) ... or actually making progress on the project ... hmm ... project it is ! |
|
@aktxyz I found that referencing that did not work. Maybe this depends on compiler or VS version? |
|
I am on the latest vs 16.10.0 ... and the relevant sections of my csproj look like this ... hope this helps |
|
@aktxyz Thank you for that information. My situation must be different since that solution doesn't fully work for me. In the case of locally referencing an analyzer that can work but when generating a Nuget package it seems required to manually specify transitient dependencies. Additionally, my use of the AdhocWorkspace type ends up meaning that compilation of the source generator fails without also adding: This is such a confusing feature of C#. It seems very difficult to use source generators due to all these issues. Hopefully this is all resolved one day by Microsoft. Source Generators also have different behavior with assembly loading/redirection when generating nugets vs local project references. It's actually an absolute headache really. |
@Turnerj Today I came across that one of my transient dependencies was .Net Standard 1.0. And this is a pain, because it is a dependency on https://www.nuget.org/packages/NETStandard.Library/1.6.1 Your solution didn't work because all projects that include a library that depends on NETStandard.Library 1.6.1 showed a lot of errors due to library version conflicts at build time. <Target Name="AddGenerationTimeReferences" AfterTargets="ResolvePackageDependenciesForBuild">
<ItemGroup>
<None Include="@(ResolvedCompileFileDefinitions)" Pack="true" PackagePath="analyzers/dotnet/cs" />
</ItemGroup>
</Target>And it just works :) |
|
For my case, it created the correct NuGet package with all the required transient packages included in the analyzers\dotnet\cs folder. <ItemGroup>
<PackageReference Include="NSwag.CodeGeneration.CSharp" Version="13.15.9" PrivateAssets="all" />
<PackageReference Include="NJsonSchema.CodeGeneration.CSharp" Version="10.6.8" PrivateAssets="all" />
<PackageReference Include="NSwag.Core.Yaml" Version="13.15.9" PrivateAssets="all" />
</ItemGroup>
There is a slight overhead here in the form of non-private libraries, but I think this can be excluded explicitly if you need it. |
Glad you got something working for yourself @HavenDV ! In my cases with source generators, they have been local projects and are only to aid the build of a different local project rather than packing the dependencies. Still hoping for some movement on dotnet/sdk#17775 to automate all of this though AFAIK there are various blockers that are preventing progress. Fingers crossed for a .NET 7 fix! |
|
I would appreciate it if you would complete your article with this workaround for NuGet. I think it would save time for some people like me. I manually maintained a large number of transient dependencies for about a year. |
|
It would be great if someone could update the section on nuget dependencies in the cookbook to include this: |
|
Hmm. None of the above solutions are working for me. Newtonsoft is file not found, no matter what I do. |
|
Spoke too soon. What fixed it for me was explicitly adding Newtonsoft into the nupkg |
After adding I was able to debug and see that my source generator worked. Sort off, after the control exits the |
|
This bug has been open for nearly two years, are there any official updates on this issue? All of the solutions above seem to be unreliable workarounds--I haven't personally gotten them to work. |
|
I haven't had this issue since .net 6.0. You just have to use the libraries that are strictly compatible with the source generator analyzer framework. |
|
I've been having the issue myself @kamronbatman . I'm having trouble tracking what the TargetPathWithTargetPlatformMoniker actually does. In some cases it seems to work, in others, it doesn't. I'm creating a custom NuGet repo on disk, and I have a project called "My.Analyzers.Extensions" I have another project, which is an analyzer, called "My.Microservice.Builder.Analyzers" which references System.Text.Json, Microsoft.Bcl.AsyncInterfaces, and My.Analyzers.Extensions. When I add However I have a workaround that seems to work, that uses the tag, which effectively packages a dll in the package. I'm concerned that it means that if the parent library that inherits this package has a different version, this analyzer will NOT use the different version, but will continue to use the packaged version. Is that Works as Designed? Here's my workaround that only packages what you need. You just need to add 'Pack="true" ' to the PackageReference. It uses a combination of @Turnerj and @kzu's blog post: |
|
Why didn't you just fix this upstream?! |
|
I personally didn't because it requires architecture review, documentation
changes and creates a new feature that must be owned by someone at
Microsoft and I don't know where to even start with that.
…On Thu, Jul 18, 2024, 8:38 AM Arad Alvand (AmirHossein Ahmadi) < ***@***.***> wrote:
Why didn't you just fix this upstream?!
—
Reply to this email directly, view it on GitHub
<#52017 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADJXJMNZHJBEKWU2TWEZ6YTZM6ZMRAVCNFSM6AAAAABLCS33OWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZWGM4TEMRQGE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
@EdLichtman My comment was actually targeted at the maintainers. |
|
One of the other reasons I haven't offered up this fix btw, and I'd love if
you could help test this, is that I think there is already a fix.
If you add Publish="true", I think it automatically publishes it to the
package location output. I just don't know if it published it to the
analyzers output directory. You may also have to put PackageOutputPath
there.
I think it may work that way because publishing a dotnet tool automatically
does this for us as of net8 sdk, and I found a bug in which
"PrivateAssets=all" causes this same problem for the lib dlls, but simply
adding Publish=true fixes that.
I'm away from my computer this week so if you are able to test this we can
ask the team to document and maintain this as part of their test plan
moving forward maybe?
Otherwise I'll try it out next week.
So basically:
```xml
<PackageReference Include="x" PackageOutputPath="analyzers..."
Publish="true"/>
```
Along with any other attributes
…On Thu, Jul 18, 2024, 10:56 AM Arad Alvand (AmirHossein Ahmadi) < ***@***.***> wrote:
@EdLichtman <https://github.com/EdLichtman> My comment was actually
targeted at the maintainers.
—
Reply to this email directly, view it on GitHub
<#52017 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADJXJMM7ULTISA6ME54LGJTZM7JTVAVCNFSM6AAAAABLCS33OWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZWG44TANRZGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
This didn't work. Your earlier solution does copy the top level dependencies, but none of their own dependencies. |
|
For my usage, a sourge generator project with referencing System.Text.Json, referenced by another .NET 8 project directly, should have these configurations in the csproj: <ItemGroup>
<PackageReference Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Text.Encodings.Web" Version="8.0.0" GeneratePathProperty="true" PrivateAssets="all" />
<PackageReference Include="System.Text.Json" Version="8.0.4" GeneratePathProperty="true" PrivateAssets="all" />
</ItemGroup>
<PropertyGroup>
<GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>
<Target Name="GetDependencyTargetPaths">
<ItemGroup>
<TargetPathWithTargetPlatformMoniker Include="$(PKGMicrosoft_Bcl_AsyncInterfaces)\lib\netstandard2.0\Microsoft.Bcl.AsyncInterfaces.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Text_Encodings_Web)\lib\netstandard2.0\System.Text.Encodings.Web.dll" IncludeRuntimeDependency="false" />
<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Text_Json)\lib\netstandard2.0\System.Text.Json.dll" IncludeRuntimeDependency="false" />
</ItemGroup>
</Target>
|
Version Used:
MSBuild: 16.9.0.11203
.NET: 5.0.201
CSC: 3.900.21.12328 (comes with .NET 5.0.201 and used by
dotnet build), 3.900.21.16010 (comes with Visual Studio and used at least bymsbuildbut probably VS too) - All the commits between these two versionsVisual Studio: 16.9.2
Steps to Reproduce / Actual Behaviour:
Sample project to recreate issue: https://github.com/Turnerj/SourceGeneratorDependencyTest
Using .NET CLI (like we use on a CI)
dotnet buildUsing MSBuild
msbuildUsing Visual Studio
Generator Warning
Expected Behavior:
Each method builds the project.
Notes:
The sample project is a minimum example of what we use in a full project: https://github.com/RehanSaeed/Schema.NET/
I'm following the guidance listed in this discussion about how to mark the package reference: #47517 (comment)
This is also seen on the Roslyn SDK C# samples: https://github.com/dotnet/roslyn-sdk/blob/0313c80ed950ac4f4eef11bb2e1c6d1009b328c4/samples/CSharp/SourceGenerators/SourceGeneratorSamples/SourceGeneratorSamples.csproj#L13-L30
I've added the transient dependency (in my case
System.Text.Encodings.Web) as per this comment in the same discussion: #47517 (reply in thread)If I use the version
5.0.0ofSystem.Text.Encodings.Web, it does build with all methods. I've tried to debug binlogs of this but I haven't found anything explaining why it resolves correctly viadotnet buildand not via Visual Studio (ormsbuild).I don't see why I can't specifically use version
5.0.1ofSystem.Text.Encodings.Web, hopefully you can work out what is going on!The text was updated successfully, but these errors were encountered: