Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using SourceLink without .git directory #159

Closed
GeertvanHorrik opened this issue Aug 27, 2018 · 13 comments

Comments

Projects
None yet
5 participants
@GeertvanHorrik
Copy link

commented Aug 27, 2018

From what I understand, the information for the remotes is being determined based on the .git directory. However, I am trying to keep my build agents "clean" by not moving the .git stuff there. I do have the commit id and remote repository url though in the variables.

Is there a way to manually override the values (e.g. a manual or command-line version) of the tooling? The only downside I foresee is that it cannot automatically translate urls from public url to raw file urls, but I can deal with that myself as well.

@tmat

This comment has been minimized.

Copy link
Member

commented Aug 27, 2018

You should be able to use Microsoft.SourceLink.* package (the one that correspond to your source control provider) and set

<PropertyGroup>
  <EnableSourceControlManagerQueries>false</EnableSourceControlManagerQueries>
  <RepositoryUrl>{repository url}</RepositoryUrl>
</PropertyGroup>

<ItemGroup>
  <SourceRoot Include="{repository root}" RepositoryUrl="{repository url}"/>
</ItemGroup>

The URL is e.g. https://github.com/dotnet/sourcelink.

@andrewlock

This comment has been minimized.

Copy link

commented Aug 30, 2018

I'm trying to do the same, but I can't get this to work. A little more guidance as to exactly what to use would be appreciated 🙂

For example, if I have:

What should my .csproj look like?

I've tried the following (hard coding the values for now):

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>
    <PublishRepositoryUrl>true</PublishRepositoryUrl>
    <EmbedUntrackedSources>true</EmbedUntrackedSources> 
   <AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
    <EnableSourceControlManagerQueries>false</EnableSourceControlManagerQueries>
    <RepositoryType>git</RepositoryType>
    <RepositoryUrl>https://github.com/andrewlock/test</RepositoryUrl>
    <RevisionId>01234567890ABCDEF</RevisionId>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-*" PrivateAssets="All"/>
  </ItemGroup>
  
  <ItemGroup>
    <SourceRoot Include="C:\repos\test\" RepositoryUrl="https://github.com/andrewlock/test"/>
  </ItemGroup>

</Project>

But this gives warnings when built, and does not embed SourceLink urls correctly:

C:\Users\andrewlock\.nuget\packages\microsoft.sourcelink.common\1.0.0-beta-63127-02\build\Microsoft.SourceLink.Common.targets(40,5): warning : No SourceRoot items specified - the generated source link is empty. [C:\repos\test\src\testproj\testproj.csproj]

Any help would be appreciated 🙂 For what it's worth, I'm trying to run the build in Docker containers in our CI machine, and I'm hoping I can pass in the required variables at runtime e.g. /p:EnableSourceControlManagerQueries=false /p:RevisionId=0123

@Meberem

This comment has been minimized.

Copy link

commented Sep 12, 2018

@tmat I am in a similar situation where I do not have the remote URL in the git config. I tried the sample you provided and I get the same error message that @andrewlock encountered. I have tried various settings, changes, tweaks, etc. to no avail, would you be able to provide us some guidance with this?

@ChristianSauer

This comment has been minimized.

Copy link

commented Oct 7, 2018

Was there any conclusion to this?

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Today I am beginning to re-investigate this. I have written a Cake build script that automatically injects the package reference and sets the required data. You can find the script here:

https://github.com/WildGums/Orc.Analytics/blob/develop/deployment/cake/components-tasks.cake#L152

During the build, I get these warnings:

C:\Users\geert\.nuget\packages\microsoft.sourcelink.common\1.0.0-beta-63127-02\build\Microsoft.SourceLink.Common.targets(40,5): warning : No SourceRoot i
tems specified - the generated source link is empty. [C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj]

Yet it has updated my csproj correctly:

  <ItemGroup>
    <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="all" />
  </ItemGroup>
  <ItemGroup>
    <SourceRoot Include="C:\Source\orc.analytics\" RepositoryUrl="https://github.com/WildGums/Orc.Analytics" />
  </ItemGroup>

In my package, it seems to embed the info correctly:

image

Then I check the pdb files using SourceLink, and I get this output:

sourcelink print-json .\Orc.Analytics.pdb
{"documents":{}}

Any points on what could be going wrong? Could it be the wrong pdb type, or ?

This is all open source, so to repro:

  1. Clone the repository
  2. In the root, run this in powershell:
.\build.ps1 -target buildandpackage
@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Looks like I should not mess with SourceRoot at all. Leaving it away results in valid documents, so it looks like it's working with the dynamic stuff :)

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Not sure yet, looks like the tests fail:

image

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

The ms build log in diagnostic mode shows this:

SourceLinkGitHubHost
    github.com
        ContentUrl = https://raw.githubusercontent.com
SourceRoot
    C:\Source\orc.analytics\
        RepositoryUrl = https://github.com/WildGums/Orc.Analytics

Will try to figure out why it doesn't pick it up.

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

More logging info:

Target "InitializeSourceRootMappedPaths: (TargetId:77)" in file "C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\15.0\Bin\Roslyn\Microsoft.Managed.Core.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "_InitializeSourceRootMappedPathsOpt" depends on it):
Using "Microsoft.CodeAnalysis.BuildTasks.MapSourceRoots" task from assembly "C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\15.0\Bin\Roslyn\Microsoft.Build.Tasks.CodeAnalysis.dll".
Task "Microsoft.CodeAnalysis.BuildTasks.MapSourceRoots" (TaskId:40)
  Task Parameter:
      SourceRoots=
          C:\Source\orc.analytics\
                  RepositoryUrl=https://github.com/WildGums/Orc.Analytics (TaskId:40)
  Output Item(s): 
      _MappedSourceRoot=
          C:\Source\orc.analytics\
                  MappedPath=C:\Source\orc.analytics\
                  RepositoryUrl=https://github.com/WildGums/Orc.Analytics (TaskId:40)
Done executing task "Microsoft.CodeAnalysis.BuildTasks.MapSourceRoots". (TaskId:40)
Removed Item(s): 
    SourceRoot=
        C:\Source\orc.analytics\
                RepositoryUrl=https://github.com/WildGums/Orc.Analytics
Added Item(s): 
    SourceRoot=
        C:\Source\orc.analytics\
                MappedPath=C:\Source\orc.analytics\
                RepositoryUrl=https://github.com/WildGums/Orc.Analytics
Done building target "InitializeSourceRootMappedPaths" in project "Orc.Analytics.csproj".: (TargetId:77)

Target "_InitializeGitHubSourceLinkUrl: (TargetId:79)" in file "C:\Users\geert\.nuget\packages\microsoft.sourcelink.github\1.0.0-beta-63127-02\build\Microsoft.SourceLink.GitHub.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "GenerateSourceLinkFile" depends on it):
Building target "_InitializeGitHubSourceLinkUrl" completely.
Output file "|C:\Source\orc.analytics\|" does not exist.
Using "Microsoft.SourceLink.GitHub.GetSourceLinkUrl" task from assembly "C:\Users\geert\.nuget\packages\microsoft.sourcelink.github\1.0.0-beta-63127-02\build\..\tools\net461\Microsoft.SourceLink.GitHub.dll".
Task "Microsoft.SourceLink.GitHub.GetSourceLinkUrl" (TaskId:41)
  Task Parameter:SourceRoot=C:\Source\orc.analytics\
                  MappedPath=C:\Source\orc.analytics\
                  RepositoryUrl=https://github.com/WildGums/Orc.Analytics (TaskId:41)
  Task Parameter:
      Hosts=
          github.com
                  ContentUrl=https://raw.githubusercontent.com (TaskId:41)
  Output Property: _SourceLinkUrlToUpdate=N/A (TaskId:41)
Done executing task "Microsoft.SourceLink.GitHub.GetSourceLinkUrl". (TaskId:41)
Done building target "_InitializeGitHubSourceLinkUrl" in project "Orc.Analytics.csproj".: (TargetId:79)
Target "GenerateSourceLinkFile: (TargetId:80)" in file "C:\Users\geert\.nuget\packages\microsoft.sourcelink.common\1.0.0-beta-63127-02\build\Microsoft.SourceLink.Common.targets" from project "C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj" (target "CoreCompile" depends on it):
Building target "GenerateSourceLinkFile" completely.
No input files were specified.
Using "Microsoft.SourceLink.Common.GenerateSourceLinkFile" task from assembly "C:\Users\geert\.nuget\packages\microsoft.sourcelink.common\1.0.0-beta-63127-02\build\..\tools\net461\Microsoft.SourceLink.Common.dll".
Task "Microsoft.SourceLink.Common.GenerateSourceLinkFile" (TaskId:42)
  Task Parameter:
      SourceRoots=
          C:\Source\orc.analytics\
                  MappedPath=C:\Source\orc.analytics\
                  RepositoryUrl=https://github.com/WildGums/Orc.Analytics (TaskId:42)
  Task Parameter:OutputFile=obj\Release\net46\Orc.Analytics.sourcelink.json (TaskId:42)
C:\Users\geert\.nuget\packages\microsoft.sourcelink.common\1.0.0-beta-63127-02\build\Microsoft.SourceLink.Common.targets(40,5): warning : No SourceRoot items specified - the generated source link is empty. [C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj]
Done executing task "Microsoft.SourceLink.Common.GenerateSourceLinkFile". (TaskId:42)
@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

This is getting interesting. When I look in the source code for Microsoft.SourceLink.Common, and don't append the SourceRoot with a \, I get an error:

<SourceRoot Include="C:\Source\orc.analytics" RepositoryUrl="https://github.com/WildGums/Orc.Analytics" />

results in

Task "Microsoft.CodeAnalysis.BuildTasks.MapSourceRoots" (TaskId:40)
  Task Parameter:
      SourceRoots=
          C:\Source\orc.analytics
                  RepositoryUrl=https://github.com/WildGums/Orc.Analytics (TaskId:40)
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Professional\MSBuild\15.0\Bin\Roslyn\Microsoft.Managed.Core.targets(90,5): error : SourceRoot paths are required to end with a slash or backslash: 'C:\Source\orc.analytics' [C:\Source\orc.analytics\src\Orc.Analytics\Orc.Analytics.csproj]
  Output Item(s): _MappedSourceRoot= (TaskId:40)

So the source roots are correctly recognized. Maybe I will just for and see how the task behaves locally.

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Debugger attached, source root is specified:

image

@GeertvanHorrik

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

Got it working :-)

We need to specify some additional properties (figured out by decompiling Microsoft.SourceLink.GitHub.dll):

  • SourceControl
  • RevisionId

When I (dynamically) add this to my csproj:

  <ItemGroup>
    <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="all" />
  </ItemGroup>
  <ItemGroup>
    <SourceRoot Include="C:\Source\orc.analytics\" RepositoryUrl="https://github.com/WildGums/Orc.Analytics" SourceControl="git" RevisionId="72b62c068c649df27c2e85c7bb78c3edd4616d9b" />
  </ItemGroup>

It all works:

image

@tmat , why isn't the tooling falling back to the RepositoryType & RevisionId properties that NuGet (Dotnet pack) seems to understand just fine?

To view the latest (working) scripts, see this url:

https://github.com/WildGums/Orc.Analytics/blob/develop/deployment/cake/components-tasks.cake#L208

@tmat

This comment has been minimized.

Copy link
Member

commented Jun 22, 2019

I have documented the minimal required content of .git directory here:
https://github.com/dotnet/sourcelink/blob/master/docs/README.md#minimal-git-repository-metadata

If you want to avoid copying the full content of .git directory you can supply the necessary information to Source Link by creating .git/HEAD and .git/config files as described in the doc.

Note: For Source Link version 1.0.0-beta2-19270-01 and lower empty directories objects and refs are also required under .git directory. Newer Source Link packages will not require them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.