Feature : Allow project reference DLLs to be added to the parent nupkg for pack target like IncludeReferencedProjects in nuget.exe

[joelverhagen]

Steps

1. dotnet new --type lib two .csproj class libraries: projectA and projectB.
2. Change <TargetFramework> to <TargetFrameworks> if you don't have #3865.
3. Make projectA have a ProjectReference to projectB.
4. Add <Type>project</Type> to the <ProjectReference>.
5. dotnet pack projectA
6. Open the resulting .nupkg's lib folder

Expected

projectB.dll should be in the .nupkg along with projectA.dll

Actual

projectB is still a package reference, not a DLL included in the package.

Environment

Tried latest dev's pack target.

dotnet --info

.NET Command Line Tools (1.0.0-preview3-004056)

Product Information:
 Version:            1.0.0-preview3-004056
 Commit SHA-1 hash:  ccc4968bc3

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64

UPDATE: Please see workaround posted as comment to see how to add ProjectReferences as DLLs in the package dynamically.

[rrelyea]

Please check spec on wiki for planned behavior

[joelverhagen]

Spec: https://github.com/NuGet/Home/wiki/Adding-nuget-pack-as-a-msbuild-target

This doesn't work:

<TreatAsPackageReference>false</TreatAsPackageReference>

The output .nuspec still has projectB as a package reference and only one file: projectA.dll under lib.

[emgarten]

This is a non-trivial feature that hasn't been implemented for RC yet.

Supporting this will require potentially walking the entire project closure to determine all projects that need to be merged into the package, or reading the assets file and matching the project closure with both the project reference flags and the pack flags found in the project (since it can be set either way).

This is also impacted by build outputs not flowing transitively to parent projects yet.

[joelverhagen]

Plan of action:

1. Build out some automated tests for pack task to cover basic scenarios and detect regression.
2. Add the original PackageSpec to the lock/assets file.
3. Add any missing properties to the assets file pack needs (e.g. path to child project output assemblies).
4. Update restore (no, not pack) to collapse project references when <TreatAsPackageReference> is true.
5. Move the PackTask away from looking at child projects. Look at restore's assets file instead.

This has repercussions on:

• Restore: basically all of the data pack needs should be collected by restore an put in the assets file. Restore should be the only guy doing a walk.
• Project/Package duality: what if a child project .csproj has a <Reference>. How is this collapsed?
• What collapsing occurs? Certainly build artifacts and <PackageReference>... what about <Reference>?

[joelverhagen]

@rohit21agrawal, I partially got through consuming the assets file from in the pack task:
https://github.com/joelverhagen/NuGet.Client/tree/jver/3891

This also has some progress on getting the output DLLs of child projects (using @(ReferenceCopyLocalPaths) MSBuild items).

This branch is pretty rough so let me know if I can clarify.

[rohit21agrawal]

As per @rrelyea : #3893 (comment)

"We don't plan to enable this in dotnet pack / msbuild /t:pack in 4.0 timeframe.
We'll listen to customer feedback and consider in the future."

[rohit21agrawal]

moving to future as this is post-rtm work.

[gulbanana]

building a nupkg from multiple projects seems like a major feature to be missing :/

[rohit21agrawal]

@gulbanana thanks for the feedback. this is something that is not planned for the 4.0 RTM release, but this is something we will definitely address in a future release.

[bbowman]

@kzu 's nugetizer 3000 does this?

[zvirja]

Hi guys,

Just out of curiosity - are there any plans to proceed on this one? Currently, I need to use the dirty workaround that is MSBuild internals specifics:

<ItemGroup>
  <_PackageFiles Include="$(OutputPath)\ReferencedProjectDll.dll">
    <BuildAction>None</BuildAction>
    <PackagePath>lib\net45\</PackagePath>
  </_PackageFiles>
</ItemGroup>

[graphicsxp]

@satuday I feel your frustration. Yesterday I waded through all the responses above. For my project, one of the suggestions above worked fine. I copied the following into my *.csproj file and dotnet pack worked like a charm. it didn't even feel like a workaround.

  <!--
    The following solves the problem that 'dotnet pack' does not include the DLLs from referenced projects.
    See https://github.com/NuGet/Home/issues/3891 for a description of the problem
    and for newer versions / workarounds / built-in methods.
  -->
  <PropertyGroup>
    <TargetsForTfmSpecificBuildOutput>$(TargetsForTfmSpecificBuildOutput);CopyProjectReferencesToPackage</TargetsForTfmSpecificBuildOutput>
    <!-- include PDBs in the NuGet package -->
    <AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
  </PropertyGroup>
  <Target Name="CopyProjectReferencesToPackage" DependsOnTargets="ResolveReferences">
    <ItemGroup>
      <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths->WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
    </ItemGroup>
  </Target>

...and add PrivateAssets="all" to each of your <ProjectReference ...>.

This will add the referenced projects dlls to the generated package.
But if the referenced projects also have dependencies on other nuget packages, those won't be added to the current package, will they?

[graphicsxp]

Hey everyone,

First off, thank you all so much for weighing in on potential solutions for this missing experience.

From what I gathered, here is a summary I've put together:

• nuget pack currently has -IncludeReferencedProjects
• MSBuild supports <Pack> and IncludeContentInPack
• dotnet pack does not support -IncludeReferencedProjects or respect existing MSBuild functionality today.

Behavior:

A built package will include referenced projects as dependencies or as part of the package. Resulting in referenced project dependencies into the .nupkg file.

Scenarios to support:

• dotnet CLI
• PackageReference
• .nuspec & .csproj packing scenarios

There are a few aspects to this work:

1. Understanding the flows for compiled assemblies. (This issue has a documented workaround already, but we still need to consider it)
2. Understanding the flows for referenced projects.
3. Understanding the flows for trimming dependencies when merging.

While we will be looking to create a proposal for everyone to provide an opinion in the short term, I'm also interested in hearing your thoughts until then.

1. What is your expectation of this feature?
2. What does it functionally do?
3. What problem does it solve for you today?
4. When would you like to see this feature?

Thanks again for the patience.

I think it's pretty clear what people need 😏. It's no longer time for questions, the requirement is in the title of the feature and it should have been implemented a long time ago already. Can you make this happen please?

[matthewjones555]

Hey everyone,

First off, thank you all so much for weighing in on potential solutions for this missing experience.

From what I gathered, here is a summary I've put together:
...

Thank you! It's wonderful to see this issue is finally being taken seriously.

Here's my take on the feedback you've asked for:

1. What is your expectation of this feature?

I'd like to easily create a nuget package that includes dependencies from projects that I don't wish to expose as their own individual packages.

So:

Package1.nupkg

Contains all binaries from:

Project1.csproj
Project2.csproj
Project3.csproj

2. What does it functionally do?

Makes it a lot easier to create packages that have a mixture of private dependencies (that will be self contained in the package), and public dependencies (that will be exposed as other packages).

This is the largest pain point right now when creating nuget packages.

3. What problem does it solve for you today?

It will allow me to get rid of pointless public nuget packages that should not actually be public.

4. When would you like to see this feature?

This feature is long overdue. I understand you guys haven't been working on it, as it hasn't been taken seriously, but this issue is 4 years old.

It would be nice to see this work prioritised so it can be part of a release within the next year. Sooner if possible.

[kzu]

FWIW, I think this is a side-effect of a (very) bad default value for IsPackable in SDK Pack. It assumes anything you don't opt out of packing, is actually a package. And therefore everything ends up creating packages, everything ends up being dependencies instead of embedded assets, etc.

I'd change this to the way it works in NuGetizer: if you provide a PackageId, you're opting in to IsPackable=true. If you explicitly set it to true, also. But otherwise, nothing is packable and referencing those projects just ends up having them as embedded assets in the referencing (packable) project.

Then just set a property to get the legacy behavior if you depended in any way on that (i.e. LegacyIsPackableDefault or whatever).

[jzabroski]

@JonDouglas I think the one other scenario to support is a better nuget.org UI that let's people know about private dependencies. Effectively, calculate the flow, but don't list them as public dependencies. Why throw away the good data if you have it available?

Basically, for a small increase in package size, you get a wealth of useful information to build tooling around. As I've already explained, this metadata is essential for building Tools or apps that want to co-integrate with hot swappable plug-ins (runtime loaded assemblies).

As far as @kzu suggestion to go off PackageId, I think as long as the tooling is there in the Project Settings dialog in Visual Studio, then that is probably an improvement. I'm not a huge fan of adding a LegacyIsPackableDefault without also adding a target date to obsolete that flag. An alternative solution is to have Visual Studio update projects if the sln version is before this change.

[MV10]

Wow, a mere 48 hours before this thread enters its fifth year...

[voroninp]

.NET 5, 5th year... Coincidence? I don't think so =D

[ericsampson]

@JonDouglas thank you very much for working on this. I would be so happy to see it available.

[roofiq]

• MSBuild supports <Pack> and IncludeContentInPack

Hello,
could you provide any example of usage for this 'IncludeContentInPack', can't find anything.

[IanKemp]

• MSBuild supports <Pack> and IncludeContentInPack

Hello,
could you provide any example of usage for this 'IncludeContentInPack', can't find anything.

https://docs.microsoft.com/en-us/dotnet/core/tools/csproj#includecontentinpack

[roofiq]

Thanks for a quick reply @IanKemp. I was actually hoping, that it will allow MSBuild to also include referenced project (as nuget includeReferencedProjects property).
I have asp.net app migrated to packageReference and SDK, I want to pack all projects to one NuGet - can't use NuGet command for it as it is said in this document - that using msbuild -t:pack is recommended for projects migrated from packages.config.

How should I approach it? What workaround is best for this? Here I found some listed

[kzu]

Just posted on the mentioned StackOverflow a full repro and how it works already in NuGetizer with no changes to the project other than just adding a package reference. Just in case you need this right away and can't wait for a fix 😁

[HelloKitty]

I feel like this is still needed. A good example might be wanting to packet project reference dependencies into a Analyzer or Source Generator Nuget package. Currently I don't think there is a way to include the assembly files directly in the package like there is for private asset Nuget package references.

[StingyJack]

I needed to be able to produce packages (A, B) for two assemblies in the same branch where one of the packages (A) depended on another (B), but used a project reference in the solution. The way I made it work was to have a nuspec for A that had B in the dependencies section, but used a version of "SAME". The SAME value was replaced at build time with the version that was being applied to the packages and before they were packed.

I posted this with a bit more info as part of at least one issue if anyone needs it.

[kzu]

@HelloKitty with NuGetizer you can pull arbitrary files from arbitrary package references with:

<None Include="lib\netstandard2.0\Newtonsoft.Json.dll" PackageReference="Newtonsoft.Json" /> 

It would be awesome if SDK pack included that capability too, since I think for non-trivial scenarios like generators, tools and even build tasks, it's quite useful.

@StingyJack I'm thinking whether a ProjectReference="..." attribute could serve the same purpose for a project reference's packed content... If so, would you like to open an issue so I can work on it? Thanks!

[iitaiy]

@satuday I feel your frustration. Yesterday I waded through all the responses above. For my project, one of the suggestions above worked fine. I copied the following into my *.csproj file and dotnet pack worked like a charm. it didn't even feel like a workaround.

  <!--
    The following solves the problem that 'dotnet pack' does not include the DLLs from referenced projects.
    See https://github.com/NuGet/Home/issues/3891 for a description of the problem
    and for newer versions / workarounds / built-in methods.
  -->
  <PropertyGroup>
    <TargetsForTfmSpecificBuildOutput>$(TargetsForTfmSpecificBuildOutput);CopyProjectReferencesToPackage</TargetsForTfmSpecificBuildOutput>
    <!-- include PDBs in the NuGet package -->
    <AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
  </PropertyGroup>
  <Target Name="CopyProjectReferencesToPackage" DependsOnTargets="ResolveReferences">
    <ItemGroup>
      <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths->WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
    </ItemGroup>
  </Target>

...and add PrivateAssets="all" to each of your <ProjectReference ...>.

This will add the referenced projects dlls to the generated package.
But if the referenced projects also have dependencies on other nuget packages, those won't be added to the current package, will they?

You are correct, this will add only the referenced project dlls and not the dependencies of other NuGet packages.
Does anyone have a solution for it?

For example:
Project A
|----> Project B

Project B
|----> SomePackage

We will get NuGet package that includes ProjectA and ProjectB dlls, but without the dependency to "SomePackage".

[AraHaan]

I think I know of a dirty hack:

1. create a dummy project (no code on it)
2. ProjectReference all the other projects you want (to ensure they get built 1st) before attempting to build the package.
3. With the dummy project made mark all of the other projects non-packable
4. have the dummy project target all of the TargetFrameworks of the other projects, they must then all match
5. Add the following to the dummy project:

<ItemGroup>
  <_PackageFiles Include="[path to each project output path]\$(TargetFramework)\[project name here].dll">
    <BuildAction>None</BuildAction>
    <PackagePath>lib\$(TargetFramework)\</PackagePath>
  </_PackageFiles>
  <_PackageFiles Include="[path to each project output path]\$(TargetFramework)\[project name here].xml">
    <BuildAction>None</BuildAction>
    <PackagePath>lib\$(TargetFramework)\</PackagePath>
  </_PackageFiles>
  <!-- add reference assemblies optionally?. -->
  <_PackageFiles Include="[path to each project output path]\$(TargetFramework)\ref\[project name here].dll">
    <BuildAction>None</BuildAction>
    <PackagePath>ref\$(TargetFramework)\</PackagePath>
  </_PackageFiles>
  <!-- add docs to reference assemblies too?. -->
  <_PackageFiles Include="[path to each project output path]\$(TargetFramework)\[project name here].xml">
    <BuildAction>None</BuildAction>
    <PackagePath>ref\$(TargetFramework)\</PackagePath>
  </_PackageFiles>
  <!-- include pdbs and source code by somehow manually making an snupkg file using all of the above
       project information ^. -->
  <!-- any other build artifacts that you want to include in the AIO package like msbuild task files
       to an msbuild task project you optionally want to include with the other projects so consumers
       can use the task too. -->
</ItemGroup>

The issue is marking the dummy csproj file as something that must not be compiled but to basically only generate an nuspec to use for packing.
Just to generate a nuspec that properly contains the dependencies of each project, and everything.

Another option is to possibly manually add the dependencies to each project into the dummy project, then skip step 2 to ensure each project you want embeeded into the package does not sneak into the Dependencies of the single package, and basically have it in an standalone solution file that then is used to make the package.

[StingyJack]

Everyone is coming up with elaborate ways to avoid having to create a 15-20 line XML file (nuspec) that will do exactly as you instruct it to do.

[matthewjones555]

What everyone wants is a tidy solution where they only have to specify things in one place. The problem with going back to nuspec files is they're a second source of truth, and can diverge from how things are set up in your projects. It's not a case of avoiding having an extra 20 line file, it's a case of avoiding having a file that can become stale with no easy way of automatically detecting the problem.

The nuget integration into msbuild is great, but it has some rather large holes, for what seems to be fundamental functionality. When you run into these problems, it makes the integration seem incomplete.

[roofiq]

@StingyJack true that. I was also searching a lot and I find creating nuspec the easiest way.
Just maybe one question. Let's say we have project with quite a lot dependencies. After build, all the DLLs and files are being created in obj and bin files.
What I have found, that the easiest solution to create nuget is to just add this element, that would simply copy folder, that is normally being published as artifact.

  <files>
    <file src="obj\Release\Package\PackageTmp\**" target="" />
  </files>

With this one, we can replace using build artifacts with nugets (in example of azure devops).
Any comments on doing it this way? What can go wrong? :)

[StingyJack]

@roofiq - the obj folder is for temporary compilation and other transient stuffs, so its a bad idea to try to use anything from an obj folder. The bin\Release folder is where you want to get the files if you are making a normal nuget.

Yer also missing a target value and are probably pulling in way too many files.

I'll usually use file elements like this so I can get the MyProject.DLL, MyProject.PDB, and MyProject.XML all packed up to support some debugging of the package and intellisense for VS. (make sure to set either "Full" or "Embedded" PDB for the Release configuration.)

<files>
    <file src="bin\Release\MyProject.*" target="net462" />
</files>

With this one, we can replace using build artifacts with nugets (in example of azure devops).
Any comments on doing it this way? What can go wrong? :)

You are making it hard for yourself in a few ways. You cant deploy a nuget package from a release pipeline with the easy to use, built in tooling. Also programs are not packages. Except if you are actually making Chocolatey packages (that use the nupkg format but are still not nuget packages) and using chocolatey to install apps. I dont get the appeal of using a nuget package format and all the rules around packing it instead of an artifact (a plain zip of the published projects from the build).

You probably also lose all of the artifact retention stuff that happens with a release pipeline by not using artifacts.

[roofiq]

@StingyJack - thanks for the long answer 👍
I know, that the final compiled files are in /bin folder. But I have investigated the actual SiteRoot folder and it was 1:1 as the one in /obj (.net framework 4.8 app). So I have decided to use this one. It actually worked.

You are making it hard for yourself in a few ways. You cant deploy a nuget package from a release pipeline with the easy to use, built in tooling.

Basically the point of my answer, was that we want to switch from using build artifacts to artifact feeds. So my task was to come up how we can easily pack project, that has mutliple dependencies.

So to sum up, if we would simply point bin/Release to be copied to Target directory specified in Nuspec file, it will be done in good practise?

[AraHaan]

@roofiq that depends on how your build setup is, if the build is complex where you need to package like:

1. 10+ projects into an single Package, this being named your own Sdk.

The crazy shit is that somehow the files to the .NET Core SDK binaries and runtimes are all packed into a single large ass package, how the f**k did they do it when we cant simply replicate that shit.

This all feels like defacto b******t to me that they can do it, yet we cannot easily do the same. All I want is to provide all my projects into a single package instead of the 10+ packages they are now that I am now having issues with maintaining the shit and then trying to find time for other development work on other things as well.

But like man the hell is wrong with the fact that .NET Core can do this on their packages, yet we cannot.

There even lacks the documentation on how you can make your own dotnet sdk that one can install and then it basically then auto references the binaries you made as well as then providing the original .NET sdk references based on the TFM just by changing the 1st line in an csproj to your own sdk. It's a lot better than just downloading and trying to maintain 10+ nuget package as that becomes a single package, plus then could be concentrated into an single git repository, which then in turn can be used to not only save time but also be used to file issues for each project within the single repo, prs, and also dependabot would update every project's dependency at once compared to 10+ projects x amount of dependencies and then worry about 100+ pull requests and waiting over 5 hours just for them all to auto merge and crap when that time could be reduced to 10 minutes from only having a single repository and generating an single package instead that holds all of the compiled files (including symbol packages with everything).

[StingyJack]

So to sum up, if we would simply point bin/Release to be copied to Target directory specified in Nuspec file, it will be done in good practise?

@roofiq - Yes, but it is going to depend on if you are packaging assemblies or programs in this way. When you refer to build artifacts, those are usually going to be programs (Web Apps, Windows desktop apps, Web Sites, WPF, AWS Lambda, Azure Function/webjob, etc.) and sometimes other things like config files. Making these into nuget packages adds complexity because you go from a simple zipped build artifact that can be unzipped and xcopied for deploying to a nupkg (also zipped, but a container with a very specific format that you need to try to make your application fit into and then extract back out of). I'm thinking you may mean programs because the path that you were getting the files from looks like where Published apps go when you dont specify a publish path.

If you had been putting your dll's by themselves into build artifacts than the answer to your question is definitely Yes, point to the bin\Release and pack/push. Just dont forget to generate and pack PDB's and XML comment files in there too (these are not required but this is far easier to do than to setup and configure symbol servers, add symbol publishing to a pipeline, then update every dev;'s visual studio instance to use the symbol server).

this being named your own Sdk

@AraHaan - I'm actually at a loss for understanding what you mean by "SDK", an example may help, but please open a different issue for what you need cos it doesnt seem aligned with this topic. Feel free to tag me. The large packages you see are actually metapackages and dont contain files.

[rtaylor72]

Sad we are now on .Net 5 and this is still an issue... I do not see any progress on this...

[AraHaan]

I found something that works however it would need some edits to work for satellite assemblies too.

https://github.com/Elskom/Sdk/
Take a look at the dummy nuspec with an csproj just to make it all work with dotnet build / dotnet pack.

[StingyJack]

@AraHaan - those nuspec files are really, really over-worded.

Rather than tangent this thread further I'm going to open an issue on the repo you linked to share some ideas with you that may reduce your maintenance pains. I'll link it here for anyone who wants to follow.