Fix Publish with GeneratePackageOnBuild=true

[wli3]

Description

The previous change https://github.com/dotnet/sdk/pull/3255/files caused the Build target to no longer be part of the dependency of the Publish target when GeneratePackageOnBuild=true. With this change, we edited the dependency targets graph to add the targets publish needs without adding _Build back to the list. We also added test coverage for the combination of related msbuild properties to prevent future regression.

Customer Impact

dotnet publish fails when GeneratePackageOnBuild=true

Regression?

Yes. By https://github.com/dotnet/sdk/pull/3255/files, since dotne core SDK 3.0.100-preview 6

Risk

Low risk.

Test changes in this PR

Added GeneratePackageOnBuild + PackAsTool + Publish/Build/Pack all combinations in unit test.

===============

Fix #3367

===============

What I did is essentially write a different dependency graph in GeneratePackageOnBuild = true to avoid circular dependency . Original bug. I added the https://github.com/dotnet/sdk/pull/3255/files and essentially regressed NuGet/Home#7801. Instead of the logic is in nuget, it is in SDK now. Publish should not skip Build if GeneratePackageOnBuild is true. Or it will mean no build even use Publish verb.

But revert 3255 means, #2114 will be back. The solution is to break Pack’s dependency on Build in GeneratePackageOnBuild. But it still need Publish. So I need to let Pack depend directly on no build version of publish.

Why cannot directly use “_PublishNoBuildAlternative”?

We want to exclude Build step regardless NoBuild flag is true or not.

Test coverage

I have GeneratePackageOnBuild + PackAsTool + Publish/Build/Pack all combinations covered in tests. Although I did discover GeneratePackageOnBuild+Pack #3471 does not work. But it was there since 2.0. And without nuget help, we cannot easily fix it.

[wli3]

i'll wait for Monday in ask mode. Don't track that

[wli3]

@nguerrera @peterhuene this is the PR i mentioned in stand up.

I tested $(GeneratePackageOnBuild) + $(PackAsTool) + Publish/Build/Pack combination, however, $(NoBuild) started to fail. And at this point I really have no confidence this change will be safe. The biggest problem is, I am essentially keep changing the dependency, until all tests pass. I cannot come up with a "grand theory" to avoid the circular dependency when _GeneratePackageOnBuild is true.

If we do nothing, we are at the starting point of 3.0 work. So no new regression from 3.0. But NuGet/Home#7801 , #3471 will be there forever. I don't think they are critical bugs, consider they could be workaround easily. But we do need a better error message when they happen.

Or you might come up with the "grand theory" to avoid the circular dependency?

[wli3]

@peterhuene @nguerrera what do you think

[peterhuene]

Hmm. Let me investigate this to see if I can offer any insights. Sorry for the delayed response.

[peterhuene]

I wonder if we should set GeneratePackageOnBuild globally to false for dotnet pack.

Your changes make sense to me for dotnet build and dotnet publish, and will work for various combinations of --no-build, GeneratePackageOnBuild, and PackAsTool.

The problem is dotnet pack. If GeneratePackageOnBuild is set to true in the project file, no build occurs for the Pack target per the NuGet changes (which is desirable for dotnet build and dotnet publish). But we expect dotnet pack to build by default regardless of GeneratePackageOnBuild. Having the CLI dotnet pack command set GeneratePackageOnBuild to false would force the Pack target to depend on Build.

Note that with these changes, dotnet pack /p:GeneratePackageOnBuild=false and dotnet pack --no-build /p:GeneratePackageOnBuild=false produce the desired behavior all around.

This really is a cluster.

@nguerrera your take?

[wli3]

@peterhuene this should fix pack + GeneratePackageOnBuild=true, but it will not fix publish + GeneratePackageOnBuild=true ?

[wli3]

@peterhuene or you propose current PR change + dotnet pack /p:GeneratePackageOnBuild=false ?

The current PR will break nobuild=true + packastool :( I need to make it more strange (do similar stuff as GeneratePackageOnBuild for NoBuild )

[peterhuene]

How is publish with GeneratePackageOnBuild=true broken for you?

Additionally, how will it break --no-build and PackAsTool=true? I get the expected behavior.

[peterhuene]

Oh wait, I see the publish behavior now with your changes.

Scratch that, I didn't have your changes any longer as I was in the middle of making changes for another PR. After rebuilding with your changes, I get the expected behavior for GeneratePackageOnBuild=true for publish.

So what doesn't work with your fix regarding publish?

I was suggesting that we use your fix plus a change to dotnet pack.

[wli3]

How is publish with GeneratePackageOnBuild=true broken for you?

This is the bug #3367. I added the https://github.com/dotnet/sdk/pull/3255/files and essentially regressed NuGet/Home#7801.

[wli3]

https://github.com/dotnet/sdk/pull/3255/files is to fix #2114

[peterhuene]

Yes, I understand that.

Perhaps I didn't make it clear that your changes seem fine, but dotnet pack is broken with GeneratePackageOnBuild=true in the project file. My suggestion to explicitly disable it for dotnet pack (i.e. set GeneratePackageOnBuild to false as a global property) would be in addition to your changes.

[wli3]

with the following diff, it seems fixed the NoBuild error. But I feel after all these changes, the dependency code is too complicated.

diff --git a/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets b/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets
index b09744f6..6b27441a 100644
--- a/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets
+++ b/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets
@@ -27,8 +27,8 @@ Copyright (c) .NET Foundation. All rights reserved.
   <PropertyGroup>
     <_ToolsSettingsFilePath>$(BaseIntermediateOutputPath)DotnetToolSettings.xml</_ToolsSettingsFilePath>
     <SuppressDependenciesWhenPacking Condition=" '$(PackAsTool)' == 'true' ">true</SuppressDependenciesWhenPacking>
-    <_PackToolPublishDependency Condition=" '$(GeneratePackageOnBuild)' != 'true' and $(IsPublishable) == 'true' ">_PublishBuildAlternative</_PackToolPublishDependency>
-    <_PackToolPublishDependency Condition=" '$(GeneratePackageOnBuild)' == 'true' and $(IsPublishable) == 'true' ">$(_PublishNoBuildAlternativeDependsOn)</_PackToolPublishDependency>
+    <_PackToolPublishDependency Condition=" ('$(GeneratePackageOnBuild)' != 'true' and '$(NoBuild)' != 'true') and $(IsPublishable) == 'true' ">_PublishBuildAlternative</_PackToolPublishDependency>
+    <_PackToolPublishDependency Condition=" ('$(GeneratePackageOnBuild)' == 'true' or '$(NoBuild)' == 'true') and $(IsPublishable) == 'true' ">$(_PublishNoBuildAlternativeDependsOn)</_PackToolPublishDependency>
   </PropertyGroup>

   <Target Name="PackTool" DependsOnTargets="GenerateToolsSettingsFileFromBuildProperty;$(_PackToolPublishDependency);_PackToolValidation" Condition=" '$(PackAsTool)' == 'true' ">

[nguerrera]

I think Peter's suggestion could work. At this point, if 3.0 has not regressed anything, given that we are getting late in 3.0 and that this is super confusing, I would suggest deferring the change.

The problem I see with dotnet pack setting a global property to fix something.

1. It will not fix msbuild /t:Pack
2. It will not fix VS Build -> Pack

@rainersigwald Do you have any ideas here?

[peterhuene]

Without Will's changes, I think there are multiple regressions in 3.0:

1. dotnet publish with GeneratePackageOnBuild=true in the project file doesn't cause a build (regression in our targets caused by Fix generate package on build pack as tool #3255). Fixed with these changes while also maintaining a fix for Project with PackAsTool and GeneratePackageOnBuild set to true fails to build #3253.
2. dotnet pack with GeneratePackageOnBuild=true in the project file doesn't cause a build (regression in the NuGet targets caused by Do not set NoBuild with GeneratePackageOnBuild because property reuse breaks publish NuGet/NuGet.Client#2822, but that change fixed dotnet publish with GeneratePackageOnBuild=true). Not fixed with these changes. Proposed fix would be to force the property to false so that the Pack target performs the build, but I'm open to alternatives (I don't like global properties either).

I can live without fixing it for msbuild /t:Pack (not ideal), but if VS invokes the Pack target directly then I think that would kibosh the proposed global property fix.

[nguerrera]

Ah, I asked yesterday if there were regressions, and the answer was no, but nevermind what I said if there are. The regressions should be fixed!

[wli3]

Not really, if we do nothing, there is no extra regression since 3.0

dotnet publish with GeneratePackageOnBuild=true in the project file doesn't cause a build (regression in our targets caused by #3255). Fixed with these changes while also maintaining a fix for #3253.

#3253 regressed NuGet/Home#7801, which is a 3.0 bug itself.

Tested in 2.2.401

dotnet pack with GeneratePackageOnBuild=true in the project file doesn't cause a build (regression in the NuGet targets caused by NuGet/NuGet.Client#2822, but that change fixed dotnet publish --no-build with GeneratePackageOnBuild=true). Not fixed with these changes. Proposed fix would be to force the property to false so that the Pack target performs the build, but I'm open to alternatives (I don't like global properties either).

Tested in 2.2.401. it was there in 2.2 at least

[peterhuene]

I thought for sure at least the dotnet pack behavior was a regression, but Will is right, it is not. I'd argue that we should at least fix the regression of the 3.0 fix for dotnet publish since we've had multiple preview releases with the fix in place now.

GeneratePackageOnBuild really doesn't seem like it is a functional feature if the 2.x behavior was to break dotnet pack and dotnet publish when set.