New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Portable PDB implementation into master #3891

Merged
merged 1 commit into from Jul 14, 2015

Conversation

Projects
None yet
6 participants
@tmat
Member

tmat commented Jul 10, 2015

Ports Portable PDB to master. Portable PDBs are now available thru /features:pdb=portable or /features:pdb=embedded.

@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat
Member

tmat commented Jul 10, 2015

@tmat tmat changed the title from Merge Portable PDB implementation into master under #ifdef to Merge Portable PDB implementation into master Jul 13, 2015

tmat added a commit that referenced this pull request Jul 14, 2015

Merge pull request #3891 from tmat/PortablePdb2
Merge Portable PDB implementation into master

@tmat tmat merged commit 4c0cb3e into dotnet:master Jul 14, 2015

1 check passed

default Build finished. No test results found.
Details

@tmat tmat deleted the tmat:PortablePdb2 branch Jul 14, 2015

@ghuntley

This comment has been minimized.

Show comment
Hide comment
@ctaggart

This comment has been minimized.

Show comment
Hide comment
@ctaggart

ctaggart Jul 23, 2015

Contributor

The PDB format supports source indexing. Will the Portable PDB format support source indexing?

Contributor

ctaggart commented Jul 23, 2015

The PDB format supports source indexing. Will the Portable PDB format support source indexing?

@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat

tmat Jul 23, 2015

Member

@ctaggart Yes. The details are still TBD.

Member

tmat commented Jul 23, 2015

@ctaggart Yes. The details are still TBD.

@ctaggart

This comment has been minimized.

Show comment
Hide comment
@ctaggart

ctaggart Jul 23, 2015

Contributor

If you want my feedback, please reach out. I'm a .NET MVP. I helped make SourceLink over the last two years to automate source indexing. http://ctaggart.github.io/SourceLink/

My wish list:

  • only support for http/https
  • authentication that would be compatible with GitHub
  • be able to navigate to all source files at design time too
Contributor

ctaggart commented Jul 23, 2015

If you want my feedback, please reach out. I'm a .NET MVP. I helped make SourceLink over the last two years to automate source indexing. http://ctaggart.github.io/SourceLink/

My wish list:

  • only support for http/https
  • authentication that would be compatible with GitHub
  • be able to navigate to all source files at design time too
@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat

tmat Jul 24, 2015

Member

@ctaggart Your feedback would indeed be welcome. The features you mention seem out of scope of the format. They are more asks for the tools that work with pdbs. The format just needs to contain enough information to support these tools.

Right now I'm thinking about adding a compiler parameter that specifies a file with additional custom debug information data. Something like /customDebugInfo:{GUID}=<path>. The compiler would add a row to CustomDebugInformation table (with Parent = Assembly) and embed the content of the specified file on the #Blob heap of the portable PDB.

In addition, we are also thinking about the ability to mark input source files as "debuggable tool generated file" (i.e. not checked into any source control, but still something that the user would want to step thru). The content of such files would be zip compressed and stored in the Portable PDB as well.

Each kind of source server (GIT, TFS, Subver, etc) has their own format. A source server specific tool would need to run as a task before compilation and create the CDI file passed to /customDebugInfo.

Since the SHA1 hashes of all source files are already included in the Document table the only information needed to find the sources in GIT repo would be the URL of the repo. So the generated CDI would just contain that.

Sounds reasonable?

Member

tmat commented Jul 24, 2015

@ctaggart Your feedback would indeed be welcome. The features you mention seem out of scope of the format. They are more asks for the tools that work with pdbs. The format just needs to contain enough information to support these tools.

Right now I'm thinking about adding a compiler parameter that specifies a file with additional custom debug information data. Something like /customDebugInfo:{GUID}=<path>. The compiler would add a row to CustomDebugInformation table (with Parent = Assembly) and embed the content of the specified file on the #Blob heap of the portable PDB.

In addition, we are also thinking about the ability to mark input source files as "debuggable tool generated file" (i.e. not checked into any source control, but still something that the user would want to step thru). The content of such files would be zip compressed and stored in the Portable PDB as well.

Each kind of source server (GIT, TFS, Subver, etc) has their own format. A source server specific tool would need to run as a task before compilation and create the CDI file passed to /customDebugInfo.

Since the SHA1 hashes of all source files are already included in the Document table the only information needed to find the sources in GIT repo would be the URL of the repo. So the generated CDI would just contain that.

Sounds reasonable?

@ctaggart

This comment has been minimized.

Show comment
Hide comment
@ctaggart

ctaggart Jul 24, 2015

Contributor

That may work. Right now, adding the source index to a pdb is done as a step after compilation. The advantage is that you can then get a list of files that the compiler has added symbols for and only index them. It would be ok to add a list of all the compiled files. After the ppdb is created, it would be useful to have another hook for validation of the source index.

Today, the source index is missing a way to specify the HTTP Authentication Scheme. It works with Basic Auth and Windows Auth. There is no way to set something so that srcsrv.dll sends the credentials (.NET .PreAuthenticate = true). This prevents GitHub compatibility with Basic Auth and is the reason why I made SourceLink-Proxy. It would be great if there was a way to specify the HTTP Auth in an agreed upon Auth workflow the tools/websites can support.

Contributor

ctaggart commented Jul 24, 2015

That may work. Right now, adding the source index to a pdb is done as a step after compilation. The advantage is that you can then get a list of files that the compiler has added symbols for and only index them. It would be ok to add a list of all the compiled files. After the ppdb is created, it would be useful to have another hook for validation of the source index.

Today, the source index is missing a way to specify the HTTP Authentication Scheme. It works with Basic Auth and Windows Auth. There is no way to set something so that srcsrv.dll sends the credentials (.NET .PreAuthenticate = true). This prevents GitHub compatibility with Basic Auth and is the reason why I made SourceLink-Proxy. It would be great if there was a way to specify the HTTP Auth in an agreed upon Auth workflow the tools/websites can support.

@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat

tmat Jul 25, 2015

Member

@ctaggart Rewriting PDB and adding more data after the fact is problematic. When the compiler builds in deterministic mode it calculates the MVID and PDB ID as a hash of the PE and PDB files, respectively. These values are then saved in the file. Any changes in the content would require rehashing of both PE and PDB.

I actually don't understand why the current source information includes a map of all source files to source control URIs. To be able to find a source file on a TFS/sd server the only information necessary is the server URL and a workspace mapping.

Member

tmat commented Jul 25, 2015

@ctaggart Rewriting PDB and adding more data after the fact is problematic. When the compiler builds in deterministic mode it calculates the MVID and PDB ID as a hash of the PE and PDB files, respectively. These values are then saved in the file. Any changes in the content would require rehashing of both PE and PDB.

I actually don't understand why the current source information includes a map of all source files to source control URIs. To be able to find a source file on a TFS/sd server the only information necessary is the server URL and a workspace mapping.

@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat

tmat Jul 25, 2015

Member

@ctaggart I have created an issue to track this: #4119

Member

tmat commented Jul 25, 2015

@ctaggart I have created an issue to track this: #4119

@erik-kallen

This comment has been minimized.

Show comment
Hide comment
@erik-kallen

erik-kallen Jul 25, 2015

Sorry for asking a stupid question, but why isn't all source code included in the PDB? Sure it will be bigger, but I would absolutely prefer that over needing to find the exact correct version of all source files, especially when they're third-party dependencies.

erik-kallen commented Jul 25, 2015

Sorry for asking a stupid question, but why isn't all source code included in the PDB? Sure it will be bigger, but I would absolutely prefer that over needing to find the exact correct version of all source files, especially when they're third-party dependencies.

@tmat

This comment has been minimized.

Show comment
Hide comment
@tmat

tmat Jul 26, 2015

Member

@erik-kallen Because it would be bigger :) We might provide an option to do so, if you want to.

Member

tmat commented Jul 26, 2015

@erik-kallen Because it would be bigger :) We might provide an option to do so, if you want to.

@0x53A

This comment has been minimized.

Show comment
Hide comment
@0x53A

0x53A Aug 13, 2015

+1 for the suggestion of erik-kallen to (optionally!) embed the source in the pdb.
This would be especially useful for small apps, which might (heavens forbid!) not even be under source control. Just saving the .exe and .pdb would then allow source-debugging.

It might also be useful for nugget libraries. Instead of having to configure 1) nuget, 2) symbol server and 3) source server, you could skip 3. (Some nuget packages also contain the pdb, so these could be self-contained, but that practice might not be optimal, as it increases the size of the package)

Lastly, it might be useful for offline scenarios, where you have the pdb but no access to the internet to download the sources.

0x53A commented Aug 13, 2015

+1 for the suggestion of erik-kallen to (optionally!) embed the source in the pdb.
This would be especially useful for small apps, which might (heavens forbid!) not even be under source control. Just saving the .exe and .pdb would then allow source-debugging.

It might also be useful for nugget libraries. Instead of having to configure 1) nuget, 2) symbol server and 3) source server, you could skip 3. (Some nuget packages also contain the pdb, so these could be self-contained, but that practice might not be optimal, as it increases the size of the package)

Lastly, it might be useful for offline scenarios, where you have the pdb but no access to the internet to download the sources.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment