Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New project system doesn't copy PDBs from packages #1458

Open
davkean opened this issue Jul 27, 2017 · 163 comments
Open

New project system doesn't copy PDBs from packages #1458

davkean opened this issue Jul 27, 2017 · 163 comments
Milestone

Comments

@davkean
Copy link
Member

@davkean davkean commented Jul 27, 2017

From @bording on July 27, 2017 3:12

When a NuGet package includes PDBs in its lib folder, the legacy project system copies the PDB as well as the assembly into the build output folder.

The new project system only copies the assembly.

We've recently added source link support , and when using the new project system with a .NET Core target, everything works fine since the PDB is referenced correctly from the global NuGet package folder.

However, because the PDB isn't copied for a .NET Framework target, we don't get any of the benefit of our sourcelinked PDB.

It would be great if the new project system would behave like the legacy project system in this case, and also copy the PDB into the project's build output folder.

Copied from original issue: dotnet/project-system#2667

EDIT: We are also using this issue to track copying XML files

@wli3
Copy link
Contributor

@wli3 wli3 commented Sep 22, 2017

wli3@0fefc41
an end to end repo/test for this bug

@wli3
Copy link
Contributor

@wli3 wli3 commented Sep 22, 2017


decide what to copy local, there should be a FileGroup called something like "DebugSymbol". However, FileGroup are from NuGet

https://github.com/NuGet/NuGet.Client/blob/efae789c75899ca0bdd063f34a4b73cd41510d56/src/NuGet.Core/NuGet.ProjectModel/LockFile/LockFileTargetLibrary.cs#L27

We need NugGet.ProjectModel to tell they are debug symbol

@dsplaisted
Copy link
Member

@dsplaisted dsplaisted commented Sep 26, 2017

If we add a FileGroup for .pdbs to the asset file, it probably makes sense to add one for the .xml documentation files too.

@dasMulli
Copy link
Contributor

@dasMulli dasMulli commented Sep 26, 2017

Maybe even for everything that msbuild considers to be related file extensions:

private string[] _relatedFileExtensions = new string[] { ".pdb", ".xml", ".pri" };
@bording
Copy link

@bording bording commented Oct 27, 2017

@wli3 Is there any more progress on this?

@onyxmaster
Copy link

@onyxmaster onyxmaster commented Nov 14, 2017

Really would like this being fixed. Having dependent PDBs is crucial for in-production diagnostics + debugging.

@jnm2
Copy link

@jnm2 jnm2 commented Nov 14, 2017

I figured out a workaround which can be pasted into the csproj:

  <!-- https://github.com/dotnet/sdk/issues/1458 -->
  <PropertyGroup>
    <AllowedReferenceRelatedFileExtensions>.pdb</AllowedReferenceRelatedFileExtensions>
  </PropertyGroup>
  <Target Name="AddReferenceRelatedPathsToCopyLocal" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <ReferenceCopyLocalPaths Include="@(_ReferenceRelatedPaths)" />
    </ItemGroup>
  </Target>
@jnm2
Copy link

@jnm2 jnm2 commented Nov 29, 2017

Heads up: this workaround will be broken by a change which has just been merged: #1725 (comment)

@nguerrera wants to fix this issue in the same release so I'll wait and see.

@bording
Copy link

@bording bording commented Dec 19, 2017

Any status update on this? I see it's now been moved to the 2.2.0 milestone.

@nguerrera There is no milestone on #1725, but am I correct in thinking that those changes haven't been included in a release yet?

@nguerrera
Copy link
Member

@nguerrera nguerrera commented Dec 19, 2017

@bording You are correct. They have been included only in unofficial 2.2.0 daily builds.

Small update: I am currently looking at optimization of how we resolve assets and I think I have a way to replicate the related file semantics of RAR, but using runtime assets correctly and without needing to probe on disk and without actually needing nuget changes. Depending on how things go, it is possible that 2.2.0 preview 1 will ship with #1725 but not the fix for this, but the plan or record is that we don't RTM 2.2 in that state.

@dasMulli
Copy link
Contributor

@dasMulli dasMulli commented Dec 23, 2017

If you could squeeze it in: It would be really cool to have metadata about satellite assemblies available as well. This would allow whipping up some targets to filter satellites coming from NuGet by locale. This has come up in https://github.com/dotnet/cli/issues/8060 and https://stackoverflow.com/questions/47942818/how-to-turn-off-language-specific-output-for-net-core-applications.

@nguerrera
Copy link
Member

@nguerrera nguerrera commented Dec 23, 2017

Will do.

@jnm2
Copy link

@jnm2 jnm2 commented Jan 6, 2018

Which release of VS is this currently slated for? I'm keeping an eye out so I can update instructions so that people know how to use NuGet packages that contain PDBs for source-stepping.

@nguerrera
Copy link
Member

@nguerrera nguerrera commented Jan 8, 2018

Adding @livarcocc for the question about VS release. I don't know if the schedule is set yet.

@jnm2
Copy link

@jnm2 jnm2 commented Jan 23, 2018

Is 15.6 definitely out of the question?

@nguerrera
Copy link
Member

@nguerrera nguerrera commented Jan 23, 2018

Yes, 15.6 is out of the question. This is being tracked for used to be called 2.2.0, but will actually be 2.1.300, which will not ship at the same time as 15.6.

@bording
Copy link

@bording bording commented Jan 23, 2018

@nguerrera How is the transition to the new SDK versioning scheme happening? Currently we have 2.1.4, so what's the next release going to be? Are there going to be additional 2.1 SDKs that still have the 2.0 runtime? Is it known which SDK release will be the one that includes the 2.1 runtime?

I'm trying to get a feel for how far out 2.1.300 is because not having PDBs copied is a pretty big pain point.

@dasMulli
Copy link
Contributor

@dasMulli dasMulli commented Jan 23, 2018

@bording see dotnet/designs#29 for the new versioning scheme.

@nguerrera nguerrera assigned nguerrera and unassigned wli3 Jan 23, 2018
@bording
Copy link

@bording bording commented Jan 23, 2018

@dasMulli I follow that repo, so I've seen that already, but thanks for the link!

My questions were more about things that aren't explicitly called out in that document. It implies that 2.1.300 will be the first version to include the 2.1 runtime. It also mentions 2.1.100. Would that be an intermediate feature release, or would that be a release from the servicing branch?

I'd really like to see this scheduled sooner than the SDK release tied to the 2.1 runtime, so I'm trying to see if my understanding of the new versioning scheme and the transition to it is correct.

@dominikjeske
Copy link

@dominikjeske dominikjeske commented Aug 1, 2020

I just found this thread after wasting whole day traying to figure out why debugging is not working. Generally speaking my feeling of debugging nugets is knowledge that is spread in many places and basic things like this still not working after 3 years? Some basic developer experience like this should be fixed or mentioned on MSDN.

@marcpopMSFT
Copy link
Collaborator

@marcpopMSFT marcpopMSFT commented Aug 3, 2020

I'll leave it to @dsplaisted to confirm the current workaround. From a prioritization perspectives, this is one of the items we plan on looking at once we finish our current .NET 5.0 planned work and we'll be working with our NuGet partners to enable this in the near future.

@kdblocher
Copy link

@kdblocher kdblocher commented Aug 4, 2020

@marcpopMSFT is this something that can be added to documentation somewhere? When I (we) hunt for how to solve this issue, we have to piece things together from the SDK, SourceLink, Visual Studio, and NuGet written documentation. It is quite a mess.

@marcpopMSFT
Copy link
Collaborator

@marcpopMSFT marcpopMSFT commented Aug 4, 2020

@KathleenDollard may be able to help with the documentation.

@TFTomSun
Copy link

@TFTomSun TFTomSun commented Sep 17, 2020

I figured out a workaround which can be pasted into the csproj:

  <!-- https://github.com/dotnet/sdk/issues/1458 -->
  <PropertyGroup>
    <AllowedReferenceRelatedFileExtensions>.pdb</AllowedReferenceRelatedFileExtensions>
  </PropertyGroup>
  <Target Name="AddReferenceRelatedPathsToCopyLocal" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <ReferenceCopyLocalPaths Include="@(_ReferenceRelatedPaths)" />
    </ItemGroup>
  </Target>

I tried this workaround, but it seems not to work atleast with a netcoreapp3.1 application. Is there an up to date workaround to get the pdbs of packages copied to the output?

@gitfool
Copy link

@gitfool gitfool commented Sep 17, 2020

@TFTomSun see #1458 (comment). I add the target to Directory.Build.targets.

@gitfool
Copy link

@gitfool gitfool commented Sep 17, 2020

@marcpopMSFT will the fixes only be in .NET 6, or will they also be back-ported to .NET 5 post release?

@marcpopMSFT
Copy link
Collaborator

@marcpopMSFT marcpopMSFT commented Sep 17, 2020

I've flagged it for discussion in our next triage meeting as we haven't firmly committed to this or a timeframe yet.

@dsplaisted
Copy link
Member

@dsplaisted dsplaisted commented Sep 17, 2020

Whenever we fix this, it is likely that it will be only in the latest SDK, but it would be fixed when you are targeting previous versions of .NET. So if we fix it in the .NET 6 SDK you would need that SDK, but when you have that SDK it would likely be fixed for your .NET 5 projects.

@gitfool
Copy link

@gitfool gitfool commented Sep 17, 2020

@dsplaisted but the .NET 6 SDK will be in preview for the next year and most of us will be pinned to stable, at most .NET 5 [SDK]. 😖

@TFTomSun
Copy link

@TFTomSun TFTomSun commented Sep 17, 2020

@gitfool @czd890 That workaround is for publishing only, right? What about the source link scenario with embedded pdbs in the package? I have a netcoreapp3.1 test project. I would like to debug into a package with source link support and embedded pdbs (because the package is not published on nuget.org). The dll is copied to the output, but the pdb not, therefore the source link debugging doesn't work.

@gitfool
Copy link

@gitfool gitfool commented Sep 17, 2020

@TFTomSun I use the workaround I linked to to debug step into dependent packages; using dotnet core 3.1, SourceLink with embedded pdbs, etc. Works great.

@gitfool
Copy link

@gitfool gitfool commented Sep 18, 2020

@TFTomSun maybe you're missing the subtlety - not surprising given the mess - that there are two workarounds in the comment I linked to. The first workaround is for during build and was originally only needed for .NET Framework targets but has also been needed since .NET Core 3.0+ targets. The second workaround, as specified directly in the linked comment, or with the proposed changes, is for during publish. Note however, that I found the second workaround was not needed with dotnet 3.1 sdk.

In summary, I'm only using the first workaround for during build and this is what will copy your pdb and xml files to the build output, which will enable debugging into dependent packages to work. I'm using a Directory.Build.targets with the following:

<Project>
  <!-- https://github.com/dotnet/sdk/issues/1458#issuecomment-420456386 -->
  <Target Name="_ResolveCopyLocalNuGetPackagePdbsAndXml" Condition="$(CopyLocalLockFileAssemblies) == true" AfterTargets="ResolveReferences">
    <ItemGroup>
      <ReferenceCopyLocalPaths
        Include="@(ReferenceCopyLocalPaths->'%(RootDir)%(Directory)%(Filename).pdb')"
        Condition="'%(ReferenceCopyLocalPaths.NuGetPackageId)' != '' and Exists('%(RootDir)%(Directory)%(Filename).pdb')" />
      <ReferenceCopyLocalPaths
        Include="@(ReferenceCopyLocalPaths->'%(RootDir)%(Directory)%(Filename).xml')"
        Condition="'%(ReferenceCopyLocalPaths.NuGetPackageId)' != '' and Exists('%(RootDir)%(Directory)%(Filename).xml')" />
    </ItemGroup>
  </Target>
</Project>
@TFTomSun
Copy link

@TFTomSun TFTomSun commented Sep 19, 2020

@gitfool You're right, I missed that. Thanks alot. I'll try that tomorrow

@marcpopMSFT
Copy link
Collaborator

@marcpopMSFT marcpopMSFT commented Oct 14, 2020

Tracked in NuGet/Home#9783. @zkat would ya'll be able to prioritize this prior to 6.0 timeframe in one of the 16.+ releases?

@omprakashdevnet
Copy link

@omprakashdevnet omprakashdevnet commented Oct 27, 2020

@gitfool

Thank you for posting it. I have tried that workaround and it was copying only .pdb files. But, if I remove the condition (Condition="$(CopyLocalLockFileAssemblies) == true") it is copying both .pdf and .xml files.

I could not find a nice documentation on CopyLocalLockFileAssemblies. I would like to understand what difference it makes if I do not add this condition.

amaitland added a commit to cefsharp/CefSharp that referenced this issue Nov 6, 2020
For release builds we embed project source and symbols (at least until dotnet/sdk#1458 is resolved)

Issue #3197
amaitland added a commit to cefsharp/CefSharp that referenced this issue Nov 9, 2020
For release builds we embed project source and symbols (at least until dotnet/sdk#1458 is resolved)

Issue #3197
@mrtristan
Copy link

@mrtristan mrtristan commented Nov 11, 2020

taking some notes here for future people that land in this hell. i find myself fighting this every 6 months or so and seemingly have the most correct solution so far (for my use-case)

my current reqs:

  • need this for debugging and loading docs from nuget packages for swashbuckle
  • using internal nuget packages (contain the xml docs)
  • needs to build and package cleanly on both local windows and 'nix docker to run on aws fargate
  • needs to be relatively dynamic

notes on my latest hurdles that i just figured out:

  • when using docker the default microsoft images disable something pertaining to loading xml docs, needs to be flipped back off
  • dynamically constructing paths to nuget package for copying files is (obviously) a case sensitive process when running on linux so you can't just use the props as they are, needs to be coerced to lower case as that's what nuget is doing when bringing them down.
  • new strings must be constructed when trying to access/mutate metadata properties in msbuild. see https://stackoverflow.com/a/8904902/1301349

modified PackageReference style:

<PackageReference Include="xxxxxx.Models" Version="$(CoreVersion)">
      <CopyToOutputDirectory>lib/netstandard2.1/*</CopyToOutputDirectory>
    </PackageReference>

block for performing the copying at applicable stages:

<Target Name="CopyPackages_Build" AfterTargets="Build">
    <Message Importance="high" Condition="%(PackageReference.CopyToOutputDirectory) != ''" Text="CopyPackages_Build - Copying packages from: $(NugetPackageRoot)$([System.String]::Copy('%(PackageReference.Identity)').ToLower())/%(PackageReference.Version)/%(PackageReference.CopyToOutputDirectory) to $(OutDir)" />
    <ItemGroup>
      <PackageReferenceFiles Condition="%(PackageReference.CopyToOutputDirectory) != ''" Include="$(NugetPackageRoot)$([System.String]::Copy('%(PackageReference.Identity)').ToLower())/%(PackageReference.Version)/%(PackageReference.CopyToOutputDirectory)" />
    </ItemGroup>
    <Copy SourceFiles="@(PackageReferenceFiles)" DestinationFolder="$(OutDir)" />
  </Target>

  <Target Name="CopyPackages_PrepareForPublish" BeforeTargets="PrepareForPublish">
    <Message Importance="high" Condition="%(PackageReference.CopyToOutputDirectory) != ''" Text="CopyPackages_PrepareForPublish - Copying packages from: $(NugetPackageRoot)$([System.String]::Copy('%(PackageReference.Identity)').ToLower())/%(PackageReference.Version)/%(PackageReference.CopyToOutputDirectory) to $(PublishDir)" />
    <ItemGroup>
      <PackageReferenceFiles Condition="%(PackageReference.CopyToOutputDirectory) != ''" Include="$(NugetPackageRoot)$([System.String]::Copy('%(PackageReference.Identity)').ToLower())/%(PackageReference.Version)/%(PackageReference.CopyToOutputDirectory)" />
    </ItemGroup>
    <Copy SourceFiles="@(PackageReferenceFiles)" DestinationFolder="$(PublishDir)" />
  </Target>

by all means, smack me around if this is an overly complicated solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.