Binary change not detected

[careri]

Hi!

NuGet: 0.13.0.0
OS: Windows 8.1

The following test case doesn't detected changes in a binary file.

1. Write a binary file, size: 255. I fill by looping a byte.
2. Commit the new change.
3. Change a couple of bytes in the file. No change detected when calling repo.Diff.compare

Best Regards
/Carl

        [TestMethod]
        public void BinDiffDetectionFailed()
        {
            // Init repo
            var repoName = "PatchTest";
            var di = new DirectoryInfo(repoName);

            if (di.Exists)
            {
                di.Delete(true);
            }

            di.Create();
            Repository.Init(di.FullName, false);
            var repo = new Repository(di.FullName);

            // Create a binary file containing 255 bytes
            var dataFI = new FileInfo(Path.Combine(di.FullName, "data1.bin"));

            using (var writer = new BinaryWriter(dataFI.Create()))
            {
                byte b = 0;
                for (int i = 0; i < 255; i++)
                {
                    writer.Write(b++);
                }
            }
            // Commit
            repo.Index.Stage(dataFI.FullName);
            repo.Commit("Commit1");

            // Change a couple of bytes in the file
            using (var writer = new BinaryWriter(dataFI.OpenWrite()))
            {
                var offset = 100;
                var data = new byte[] { 10, 11, 12, 13, 14 };
                writer.Seek(offset, SeekOrigin.Begin);
                writer.Write(data);
            }
            // Call diff, no diffs are found...
            var paths = new List<string>() { dataFI.FullName };
            var c = repo.Diff.Compare(paths);
            Assert.AreEqual(1, c.Count(), "No change detected");
        }

[yorah]

It seems to be related to the modification time of your file: as you modify your file almost right after its creation, and as the modifications don't change the size of the file (you change a few bytes) then (libgit2 thinks)[https://github.com/libgit2/libgit2/blob/development/src/diff.c#L736-L747] that the file has not been modified . This is a shortcut that usually works to speed up comparisons.

if you add a Thread.Sleep(TimeSpan.FromMilliseconds(1000)); right before making modifications to your file, then the repo.Diff.Compare() correctly reports the change.

Is it a real-world scenario (i.e.: modifying a newly created file while not changing its size/mode/other attributes, in a very small time lapse)?

/cc @arrbee

[careri]

Hi again!

No it's not a real-world scenario, I was just suprised it didn't catch the change. I actually did try to sleep 100ms, but that was to short.

Thanks for the info, I'll close this issue then.

/Carl

[glennslaven]

Hi, I did just have this in a real-world scenario, I was cloning then modifying a file immediately then committing and pushing. The Thread.Sleep trick didn't work for me, don't know why, but this did

File.SetLastWriteTime(filePath, DateTime.UtcNow.AddHours(-1));

[nulltoken]

The Thread.Sleep trick didn't work for me, don't know why, but this did

File.SetLastWriteTime(filePath, DateTime.UtcNow.AddHours(-1));

@glennslaven Interesting! Do you think you could produce a repro case?

[careri]

Maybe a dumb question, but did you put the Thread.Sleep() before you modified the file?
(I did put the sleep before calling repo.diff first time... that doesn't help much) :)

[glennslaven]

Have a repo with the file test.txt that contains just a DateTime.Now.Ticks value.

var credentials = new Credentials {Username = "username", Password = "password"};
Repository.Clone("https://github.com/path/to.git", gitPath, false, true, null, null, credentials);

var filePath = Path.Combine(repo.Info.WorkingDirectory, "test.txt");

File.WriteAllText(filePath, DateTime.Now.Ticks);

at this point repo.Index.RetrieveStatus("test.txt") will return FileStatus.Unaltered as, I assume, the byte size hasn't changed and the file date modified is within the same millisecond.

however adding File.SetLastWriteTime(filePath, DateTime.UtcNow.AddHours(-1)); after writing to the file works. For me, putting a Thread.Sleep(1000) before writing to the file didn't make a difference.

[nulltoken]

however adding File.SetLastWriteTime(filePath, DateTime.UtcNow.AddHours(-1)); after writing to the file works. For me, putting a Thread.Sleep(1000) before writing to the file didn't make a difference.

@yorah Can you reproduce this as well?

[yorah]

Mmm, nope :(

@glennslaven Can you confirm that the following test does not pass for you (it would be nice if you can grab a copy of lg2s source and execute the test in that context, as it uses some helper functions from there):

    [Fact]
    public void Shout()
    {
        string localRepoPath = InitNewRepository();
        using (var repo = new Repository(localRepoPath))
        {
            var filePath = Touch(repo.Info.WorkingDirectory, "test.txt", DateTime.Now.Ticks.ToString());
            repo.Index.Stage(filePath);
            repo.Commit("first commit");

            // UEven with the Thread.Sleep, the test does not pass?
            Thread.Sleep(1000);

            File.WriteAllText(filePath, DateTime.Now.Ticks.ToString());

            var status = repo.Index.RetrieveStatus("test.txt");
            Assert.Equal(FileStatus.Modified, status);
        }
    }

[AraHaan]

@nulltoken, @yorah Oddly Enough I run into this even if opening the file after restarting the process (on restart I check if their are local changes, then commit and push them). However it thinks there is no changes in this code (it skips the foreach):

var result = new StringBuilder();
foreach (var c in this.repo!.Diff.Compare<TreeChanges>())
{
    if (c.Status != ChangeKind.Unmodified)
    {
        result.Append($"{c.Status} {c.Path} {(c.Status == ChangeKind.Renamed ? c.OldPath : string.Empty)}");
    }
}

return result.ToString();

Ironically enough however when executing git status in powershell it outputs:

On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   ExpiringVerifications.json
        new file:   FilteredDatas.json

probably because it is a copy of git from git-for-windows?

Edit: After further looking it looks to me that one can do this after editing and doing git add .on code based on the wiki to stage changes (if any) first:

foreach (var c in this.repo!.Index/*.Diff.Compare<TreeChanges>()*/)
{
    if (c.StageLevel == StageLevel.Staged)
    {
        result.Append($"{c.Path}");
    }
}

return result.ToString();

Downside to this however is that doing this seems to make my code think that it is always changed even when there are no changes to some files.