TeamCity Repository doesn't have any commit - need to support object/info/alternates

[ctaggart]

I'm having trouble getting the latest beta to work from TeamCity ctaggart/SourceLink#359 . TeamCity is simply delegating to run the build from the official docker image. I'm create the standard CI environment variables.

I've set the Standard CI environment variables:

[1:11:50][Step 1/1] STANDARD_CI_REPOSITORY_TYPE: git
[21:11:50][Step 1/1] STANDARD_CI_REPOSITORY_URL: git@github.ecorp.test:taggac/vsphere-automation-sdk-.net.git
[21:11:50][Step 1/1] STANDARD_CI_SOURCE_REVISION_ID: 2faeb58bbe59b000c517f7c19e53fc371b10324e

1:12:38][Step 1/1] /root/.nuget/packages/microsoft.build.tasks.git/1.0.0-beta-63105-01/build/Microsoft.Build.Tasks.Git.targets(36,5): warning : Repository doesn't have any commit, the source code won't be available via source link. [/build/VMware.vSphere.Management/VMware.vSphere.Management.csproj]
[21:12:38][Step 1/1] /root/.nuget/packages/microsoft.sourcelink.common/1.0.0-beta-63105-01/build/Microsoft.SourceLink.Common.targets(73,5): warning : No SourceRoot items specified - the generated source link is empty. [/build/VMware.vSphere.Management/VMware.vSphere.Management.csproj]
[21:14:43][Step 1/1]   VMware.vSphere.Management -> /build/VMware.vSphere.Management/bin/release/netstandard2.0/VMware.vSphere.Management.dll

The sourcelink json ends up being {"documents":{}}, so it did not work.

[tmat]

STANDARD_CI variables have no effect on SourceLink. We do not have a source control provider yet that reads the information from STANDARD_CI variables instead of from the git repository itself.
Microsoft.SourceLink.GitHub reads the info from the git repository. Seems like the repository in the docker image is empty.

[ctaggart]

The root cause of this is that the TeamCity uses git object/info/alternates to speed up clones. These are basically links to files elsewhere. The Docker process needed access to these. git log -3 would fail from inside the docker container with:

[22:18:28][Step 1/1] error: object directory /opt/teamcity/TeamCity/buildAgent10/system/git/git-30CD863F.git/objects does not exist; check .git/objects/info/alternates.
[22:18:28][Step 1/1] fatal: bad object HEAD
[22:18:29][Step 1/1] Process exited with code 128

• https://confluence.jetbrains.com/display/TCD10/Git
• https://teamcity-support.jetbrains.com/hc/en-us/community/posts/206831685-teamcity-git-use-local-mirrors-vs-teamcity-git-useAlternates
  https://git-scm.com/docs/gitrepository-layout#gitrepository-layout-objectsinfoalternates

The solution I used was to simply mount it readonly at the same location:

docker run --rm -v $PWD:/app -w /app \
-v /opt/teamcity:/opt/teamcity:ro \
microsoft/dotnet:2.1-sdk \
./build.sh

Any advantages of settings the STANDARD_CI variables? Is the main benefit that it set DeterministicSourcePaths like we saw in JamesNK/Newtonsoft.Json#1746 (comment) ?

[ctaggart]

I can't seem to figure out where Microsoft.Build.StandardCI.props is used:
https://github.com/dotnet/sourcelink/blob/master/src/Microsoft.Build.StandardCI/build/Microsoft.Build.StandardCI.props

I was expecting ContinuousIntegrationBuild to be set if STANDARD_CI_REPOSITORY_URL is set.

<ContinuousIntegrationBuild Condition="'$(ContinuousIntegrationBuild)' == '' and '$(DesignTimeBuild)' != 'true' and '$(STANDARD_CI_REPOSITORY_URL)' != ''">true</ContinuousIntegrationBuild>

[tmat]

@ctaggart Let's reopen this issue to track support for object/info/alternates.

The root cause of this is that the TeamCity uses git object/info/alternates to speed up clones

@bording Does libgit2 automatically resolve objects in object/info/alternates or do I need to call some extra APIs to get the HEAD commit sha in case the repo uses alternates?

[bording]

@tmat I don't see anything in LibGit2Sharp that directly interacts with the alternates file, but I do see code in libgit2 itself related to it.

@ethomson will be better qualified to answer if that means libgit2 just handles it automatically, or if that means we have an API we need to expose in LibGit2Sharp.

[ethomson]

Yes, libgit2 supports alternates and there shouldn't be any additional configuration necessary to make them work. (And thus, LibGit2Sharp, too.)

[ctaggart]

git and libgit2 both work correctly with object/info/alternates. May be a better error could be thrown. I had to run git to figure out what the the problem was. The issue was that I only mapped the current directory to a docker volume, but because of object/info/alternates I needed to also map the part of the file system that they linked to. An alternative solution would be to figure out how to disable object/info/alternates on the build server, but it is used to speed up builds.

[tmat]

In SourceLink we call Repository.Head.Tip?.Sha to get the HEAD SHA. If Tip is null it reports that the repository does not have any commit.

[tmat]

@ctaggart Ok to close this issue?

[ctaggart]

A better error message would be nice, but not high priority

[tmat]

I don't know how to detect the condition. It seems indistinguishable from empty repository.