# Tree to Tree diffing issue #196

Closed
opened this Issue Jul 17, 2012 · 27 comments

None yet

### 5 participants

Member

Repository -> git://github.com/timburks/nu.git

Parent -> b46ed8403aba281d47619f71dd46fefa08e5c809

Diffing those two commits creates two entries with the same name
See timburks/nu@d7a2068

=== Stack Trace ===
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add)
C:\data\workspaces\git\libgit2sharp\LibGit2Sharp\TreeChanges.cs:line 98
at LibGit2Sharp.TreeChanges.PrintCallBack(IntPtr data, GitDiffDelta delta,
GitDiffRange range, GitDiffLineOrigin lineorigin, IntPtr content, UInt32 contentlen)
aces\git\libgit2sharp\LibGit2Sharp\TreeChanges.cs:line 53
at LibGit2Sharp.Core.NativeMethods.git_diff_print_patch(DiffListSafeHandle diff,
IntPtr data, git_diff_data_fn printCallback)
at LibGit2Sharp.TreeChanges..ctor(DiffListSafeHandle diff) in
C:\data\workspaces\git\libgit2sharp\LibGit2Sharp\TreeChanges.cs:line 41
at LibGit2Sharp.Diff.Compare(Tree oldTree, Tree newTree) in
C:\data\workspaces\git\libgit2sharp\LibGit2Sharp\Diff.cs:line 34

was assigned Jul 17, 2012
Member

/cc @carlosmn

Member
commented Jul 17, 2012

Here is the relevant libgit2 code:

https://github.com/libgit2/libgit2/blob/development/src/diff.c#L481-483

When the type of a file changes (e.g. symlink -> regular file) then two entries will be created for the path, a deletion and a addition.

Member

@arrbee Thanks a lot for this pointer.

When one gives a little thought about it, it indeed makes sense.

I'll recreate a test case as part of the fix to make sure this is covered.

Member

@carlosmn I've got a slight issue

The two following tests pass in my native Win7/x86 environment against the latest tip of vNext. According to my understanding of your issue, they should fail :-/

The first one attempts at recreating the issue.
The second one was created out of despair and runs against a fresh clone of nu.git.

[Fact]
public void CanHandleTwoTreeEntryChangesWithTheSamePath()
{
SelfCleaningDirectory scd = BuildSelfCleaningDirectory();

using (Repository repo = Repository.Init(scd.DirectoryPath))
{
Blob mainContent = CreateBlob(repo, "awesome content\n");

var tdOld = new TreeDefinition()

Tree treeOld = repo.ObjectDatabase.CreateTree(tdOld);

var tdNew = new TreeDefinition()

Tree treeNew = repo.ObjectDatabase.CreateTree(tdNew);

TreeChanges changes = repo.Diff.Compare(treeOld, treeNew);
Assert.NotNull(changes);
}
}

private static Blob CreateBlob(Repository repo, string content)
{
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(content)))
{
}
}

[Fact]
public void Issue_196()
{
using (var repo = new Repository(@"D:\Temp\sss\nu"))
{
var treeOld = repo.Lookup<Commit>("b46ed84").Tree;
var treeNew = repo.Lookup<Commit>("d7a2068").Tree;

TreeChanges changes = repo.Diff.Compare(treeOld, treeNew);
Assert.NotNull(changes);
}
}

You'll have to apply the following patch to allow the creation of a symlink:

diff --git a/LibGit2Sharp/TreeDefinition.cs b/LibGit2Sharp/TreeDefinition.cs
index 8a725a5..7da0eea 100644
--- a/LibGit2Sharp/TreeDefinition.cs
+++ b/LibGit2Sharp/TreeDefinition.cs
@@ -136,7 +136,7 @@ public TreeDefinition Add(string targetTreeEntryPath, Blob blob, Mode mode)
{
Ensure.ArgumentNotNull(blob, "blob");
Ensure.ArgumentConformsTo(mode,
-                                      m => m.HasAny(new[] { Mode.ExecutableFile, Mode.NonExecutableFile, Mode.NonExecutableGroupWritableFile }), "mode");
+                                      m => m.HasAny(new[] { Mode.ExecutableFile, Mode.NonExecutableFile, Mode.NonExecutableGroupWritableFile, Mode.SymbolicLink }), "mode");

TreeEntryDefinition ted = TreeEntryDefinition.From(blob, mode);

This libgit2 code https://github.com/libgit2/libgit2/blob/development/src/diff.c#L459-462 might explain why the tests pass on my computer. However, the stack trace you've provided looks like it's coming from a Win environment, too...

• Are you running the latest vNext? (Repository.Version outputs 0.9.5-unknown-1d94a7d (x86) on my box)
• Can you share some detail about the hosting platform where the exception was raised?
• Are you able to make the tests fail?
Member

I't not based on vNext but v9.5.0 (because the modified library). I haven't had time to test it yet, though it looks like it should show the issue.

The issue did happen on a Windows system, so it's not libgit2 working around the issue. I'll investigate today.

Member

Works for me. I.e. it fails. This is with your first two tests, so it looks like it's able to recreate the situation on Linux/Mono.

/home/carlos/git/libgit2sharp/CI-build.msbuild (default targets) ->
(Test target) ->

/home/carlos/git/libgit2sharp/CI-build.msbuild: error : LibGit2Sharp.Tests.DiffTreeToTreeFixture.CanHandleTwoTreeEntryChangesWithTheSamePath: System.ArgumentException : An element with the same key already exists in the dictionary.
/home/carlos/git/libgit2sharp/CI-build.msbuild: error :   at System.Collections.Generic.Dictionary2[LibGit2Sharp.Core.FilePath,LibGit2Sharp.TreeEntryChanges].Add (LibGit2Sharp.Core.FilePath key, LibGit2Sharp.TreeEntryChanges value) [0x00000] in <filename unknown>:0
at LibGit2Sharp.TreeChanges.AddFileChange (LibGit2Sharp.Core.GitDiffDelta delta) [0x00000] in <filename unknown>:0
at LibGit2Sharp.TreeChanges.PrintCallBack (IntPtr data, LibGit2Sharp.Core.GitDiffDelta delta, LibGit2Sharp.Core.GitDiffRange range, GitDiffLineOrigin lineorigin, IntPtr content, UInt32 contentlen) [0x00000] in <filename unknown>:0
at (wrapper native-to-managed) LibGit2Sharp.TreeChanges:PrintCallBack (intptr,LibGit2Sharp.Core.GitDiffDelta,LibGit2Sharp.Core.GitDiffRange,LibGit2Sharp.Core.GitDiffLineOrigin,intptr,uint)
at (wrapper managed-to-native) LibGit2Sharp.Core.NativeMethods:git_diff_print_patch (LibGit2Sharp.Core.Handles.DiffListSafeHandle,intptr,LibGit2Sharp.Core.NativeMethods/git_diff_data_fn)
at LibGit2Sharp.TreeChanges..ctor (LibGit2Sharp.Core.Handles.DiffListSafeHandle diff) [0x00000] in <filename unknown>:0
at LibGit2Sharp.Diff.Compare (LibGit2Sharp.Tree oldTree, LibGit2Sharp.Tree newTree, IEnumerable1 paths) [0x00000] in <filename unknown>:0
at LibGit2Sharp.Tests.DiffTreeToTreeFixture.CanHandleTwoTreeEntryChangesWithTheSamePath () [0x00000] in <filename unknown>:0
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0

Member

it's able to recreate the situation on Linux/Mono

That's a plus!

Have you had the chance to run it on Windows as well?

Member

I takes more willpower than I had to start up the Windows VM, but I have to double-check whether something is a mono bug, so I might do it later.

Member

Member

@carlosmn I've started to work on a fix

@carlosmn @arrbee I have a slight doubt, as the Status API is diff based, could it be possible that another LibGit2Sharp bug might be hiding in this area as well? Maybe with the symlink in the Tree and a regular file in the workdir... What do you think?

Member

That situation sounds like it should also have two distinct changes for the one file, so it should also be affected.

Member

@carlosmn I've worked on a first fix https://github.com/nulltoken/libgit2sharp/tree/fix/issue-196

I'm not especially happy with the code, but that should do it for now. This will require some refactoring later, though as it's barely readable now.

Could you please fill in the gaps regarding assertions on Linux and make sure the code actually works?

referenced this issue Aug 12, 2012
Closed

#### Fix handling of blob mode change #201

added a commit to dahlbyk/libgit2sharp that referenced this issue Aug 14, 2012
 nulltoken Fix handling of blob mode change Fix #196 c80a9f0
referenced this issue in libgit2/libgit2 Aug 15, 2012
Closed

Member
commented Jan 31, 2013

Not sure where we left off on this, but https://github.com/dahlbyk/libgit2sharp/compare/fix/issue196 has been rebased on latest.

It includes a standalone commit (dahlbyk@71eb036) with build.libgit2sharp.x64.cmd which might be worth cherry-picking into vNext.

Member

It includes a standalone commit (dahlbyk@71eb036) with build.libgit2sharp.x64.cmd which might be worth cherry-picking into vNext.

I can't exactly remember why we would need this. Isn't the 32bits version of msbuild able to correctly produce an AnyCPU build?

Member
commented Feb 1, 2013

I don't know that there's any difference in the actual build, but tests run in 32-bit by default:

Test:
xUnit.net MSBuild runner (32-bit .NET 4.0.30319.18010)
xunit.dll:     Version 1.9.0.1566
Test assembly: C:\Dev\GitHub\libgit2sharp\LibGit2Sharp.Tests\bin\Release\LibGit2Sharp.Tests.dll


With build.libgit2sharp.x64.cmd:

Test:
xUnit.net MSBuild runner (64-bit .NET 4.0.30319.18010)
xunit.dll:     Version 1.9.0.1566

Member

@dahlbyk I wonder if we could tweak the build.libgit2sharp.cmd batch in order to make less "32-bits only"

From what I've read, %PROGRAMFILES(x86)% is only defined on x64 platforms.

Could you see if something like this would work on your computer, please?

SETLOCAL

SET BASEDIR=%~dp0
SET FrameworkVersion=v4.0.30319
SET Suffix=

IF DEFINED %PROGRAMFILES(x86)% SET Suffix=64

SET FrameworkDir=%SystemRoot%\Microsoft.NET\Framework%Suffix%
SET CommitSha=%~1

"%FrameworkDir%\%FrameworkVersion%\msbuild.exe" "%BASEDIR%CI-build.msbuild" /property:CommitSha=%CommitSha%

ENDLOCAL

EXIT /B %ERRORLEVEL%

Member

@dahlbyk Forget about my last proposal. It might work, but wouldn't help ;-)

/cc @jamill @ben

Member

We might need to run those tests in 32 and 64 bits mode, thus against the clr2.0 and clr4.0.

• Adding the xUnit.console.*.exe runner binaries
• Replacing the *.cmd batch with a .ps1 script which would run msbuild (by specifying the Build target) then run a combination of tests depending on passed parameters
Member
commented Feb 1, 2013

Doing more than this at the moment really feels like YAGNI to me. We already have both architectures tested with CI, so this .cmd is really only useful if you're trying to debug something that works in x86 but not in x64.

Member

Doing more than this at the moment really feels like YAGNI to me. We already have both architectures tested with CI, so this .cmd is really only useful if you're trying to debug something that works in x86 but not in x64.

Fair enough. I've cherry-picked your commit and will open an issue.

Member

Not sure where we left off on this, but https://github.com/dahlbyk/libgit2sharp/compare/fix/issue196 has been rebased on latest.

I've slightly rewrote the tree-to-tree test in C# (cf. nulltoken/topic/type-change). However, I'm not really satisfied with it. I really don't like returning a IEnumerable from the TreeChanges indexer. I'd like to see if passing GIT_DIFF_INCLUDE_TYPECHANGE, and adding a new TypeChanged property could avoid this. While I'm at it, I may elaborate on @roend83's #278 and include rename detection (unless he's available so that we can collaborate).

The tree-to-workdir test still fails on Windows.

commented Feb 4, 2013

@nulltoken I'd be happy to help. Let me know what I can do.

Member

@nulltoken I'd be happy to help. Let me know what I can do.

@roend83 Thanks a lot for this! It would be awesome if you could rebase #278 on top of vNext and cover it with some tests. Maybe one identifying the exact renamings and another one identifying the copying.

Would that fit you?

commented Feb 4, 2013

@nulltoken Yeah, I'll try to get to it tonight.

added a commit to nulltoken/libgit2sharp that referenced this issue Feb 6, 2013
 nulltoken Make Diff.Compare able to expose type changes Fix #196 89bd6e4
added a commit to nulltoken/libgit2sharp that referenced this issue Feb 6, 2013
 nulltoken Make Diff.Compare able to expose type changes Partially fix #196 426a7ea
Member

added a commit to nulltoken/libgit2 that referenced this issue Feb 6, 2013
 nulltoken diff: Enhance tree-to-tree diff test coverage These tests are related to issue libgit2/libgit2sharp#196 e899345
Member
commented Feb 7, 2013

I like topic/type-change a lot better than the alternative.

Also, TreeChanges.DebuggerDisplay should totally use posh-git symbology for labeling added/modified/deleted. :)

added a commit that closed this issue Feb 7, 2013
 nulltoken Make Diff.Compare able to expose type changes Fix #196 3db50fc
closed this in 3db50fc Feb 7, 2013
Member

@carlosmn This took some time. Sorry for that. It should now be fixed.

Also, TreeChanges.DebuggerDisplay should totally use posh-git symbology for labeling added/modified/deleted. :)

@dahlbyk Deal! Could you please send a PR covering this?

This was referenced Feb 8, 2013
Closed

Closed

#### Refactor iterators and diff libgit2/libgit2#1326

added a commit to nulltoken/libgit2 that referenced this issue Feb 14, 2013
 nulltoken diff: Enhance tree_to_workdir diff test coverage This test is related to issue libgit2/libgit2sharp#196 e0e8a94
added a commit to nulltoken/libgit2 that referenced this issue Dec 21, 2013
 nulltoken diff: Enhance tree_to_workdir diff test coverage This test is related to issue libgit2/libgit2sharp#196 e574524
pushed a commit to phatblat/libgit2 that referenced this issue Sep 13, 2014
 nulltoken diff: Enhance tree-to-tree diff test coverage These tests are related to issue libgit2/libgit2sharp#196 89d171d
referenced this issue Mar 18, 2015
Merged