Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unlink of file '.git/...' failed #3336

Closed
1 task done
thany opened this issue Jul 27, 2021 · 24 comments
Closed
1 task done

Unlink of file '.git/...' failed #3336

thany opened this issue Jul 27, 2021 · 24 comments

Comments

@thany
Copy link

thany commented Jul 27, 2021

  • I was not able to find an open or closed issue matching what I'm seeing

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options

git version 2.32.0.windows.2
cpu: x86_64
built from commit: 3d45ac813c4adf97fe3733c1f763ab6617d5add5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
$ cmd.exe /c ver

Microsoft Windows [Version 10.0.19043.1110]

That's the 64 bits version, obviously.

  • What options did you set as part of the installation? Or did you choose the
    defaults?
# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VisualStudioCode
Custom Editor Path:
Default Branch Option: master
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: ConHost
Git Pull Behavior Option: Rebase
Use Credential Manager: Core
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable Pseudo Console Support: Disabled
Enable FSMonitor: Enabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?

I also use GitKraken, and this issues happens a lot while that's running, but also while it isn't (and even hasn't been sine the latest reboot). I also have VScode open all the time, which has a built-in Git client, but I'm not using it.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

CMD

git pull --rebase

But really many other commands trigger an automatic git gc, as you (authors) probably know ;)

  • What did you expect to occur after running these commands?

When git gc triggers automatically, don't complain when files can't be unlinked. As a user, I shouldn't have to care about that. Either git should ignore such problem (if it's harmless), or recover from it transparently.

  • What actually happened instead?
(after doing a successful command)

Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
Enumerating objects: 297431, done.
Counting objects: 100% (297431/297431), done.
Delta compression using up to 12 threads
Compressing objects: 100% (74044/74044), done.
Writing objects: 100% (297431/297431), done.
Total 297431 (delta 204855), reused 296282 (delta 203853), pack-reused 0
Unlink of file '.git/objects/pack/pack-2ebaef8c3398262c246a752a06git branch24602c42c80332.pack' failed. Should I try again? (y/n)

This "unlink" question repeats itself seemingly endlessly whether I answer yes or no. It's also impossible to know what the "right" answer is. Should I try again - I don't know, should you? Either way, answering yes just repeats the question for the same file. Answering no advances it to the next file, but god knows how many there are.

  • If the problem was occurring with a specific repository, can you provide the
    URL to that repository to help us with testing?

It's a general problem. Happens with multiple repo's, but most notably with very large ones. I wonder if this can happen with the repo for Git itself...

However, it does only happen after doing "some work". I'm not sure exactly when git gc triggers automatically, but it does. And I'm not sure when it decides unlinking a file cannot be done and bothers the user with it, but it will happen "at some point".

@dscho
Copy link
Member

dscho commented Jul 27, 2021

Could you try to figure out what process is holding an open handle to that .pack file, e.g. using Sysinternals' HANDLE tool?

@thany
Copy link
Author

thany commented Jul 30, 2021

This might take a while. I'd have to wait until the problem reproduces itself first 😀

In the mean time, is it really a problem if Git would simply ignore being unable to delete in-use .pack files? Does Git really need those files gone?

@dscho
Copy link
Member

dscho commented Aug 3, 2021

In the mean time, is it really a problem if Git would simply ignore being unable to delete in-use .pack files? Does Git really need those files gone?

That is an interesting idea.

Obviously, disk space would be wasted more easily this way, and if there was a consistent bug in the future that always keeps a handle to the pack(s) open while trying to clean up no-longer-needed packs, we would easily miss that bug, and disk space usage would skyrocket with every gc.

But as a band-aid, guarded by an opt-in config setting, this could be very useful.

Are you up to implementing it? I would try to find some time to provide you with enough pointers to get started.

@thany
Copy link
Author

thany commented Aug 4, 2021

I was able te reproduce the issue while GitKraken was running. I suspect if GK isn't running, it might be VScode's internal Git client causing this problem in a similar way. I've send some stern feedback GK's way, since I have proof that that's what is keeping those files locked. I haven't been able to prove the same thing for VScode, so I shouldn't be sending feedback their way just yet.

Be that as it may, I'm not actually sure where the fix should be. I'm pretty sure software like that doesn't invent totally custom Git clients, but instead are using ready-to-use packages (or even the commandline) under the bonnet. I do think it's somewhat common to use the CLI and a GUI side-by-side.

As for implementing that option: I'm sorry, but I'm not that kind of programmer. I don't think I'd be able to do a very good job. I can help testing though.

@mikekaganski
Copy link

The process holding the files is ... git itself - namely, the other instance of git.

The process tree (as seen in Process Explorer) is:

git.exe		53 556 K	43 744 K	17608	Git for Windows	The Git Development Community	"C:\Program Files\Git\mingw64\bin\git.exe" pull -r
    git.exe		35 964 K	36 600 K	23236	Git for Windows	The Git Development Community	git fetch --update-head-ok
        git.exe		2 756 K	6 996 K	20140	Git for Windows	The Git Development Community	git maintenance run --auto --no-quiet
            git.exe		2 956 K	8 192 K	14024	Git for Windows	The Git Development Community	git gc --auto --no-quiet
                git.exe		2 748 K	7 048 K	28192	Git for Windows	The Git Development Community	git repack -d -l -A --unpack-unreachable=2.weeks.ago

The topmost one holds all those handles to the files failing to unlink; the bottommost one tries to unlink them.

image

@dscho
Copy link
Member

dscho commented Sep 7, 2021

I guess it is that read_cache_unmerged() call that causes Git to open those packs, and the repack later in the call chain needs them not to be opened. The solution probably looks somewhat like this:

diff --git a/builtin/pull.c b/builtin/pull.c
index 3e13f810843..5d51c84d305 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -998,6 +998,7 @@ int cmd_pull(int argc, const char **argv, const char *prefix)
 			oidclr(&rebase_fork_point);
 	}
 
+	close_object_store(the_repository->objects);
 	if (run_fetch(repo, refspecs))
 		return 1;
 

@mikekaganski do you think you could try that?

@mikekaganski
Copy link

@mikekaganski do you think you could try that?

Yes, I can try to build, but I need help to get an exact command/sequence of actions that would trigger the GC during the pull command.

dscho added a commit to dscho/git that referenced this issue Sep 7, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes git-for-windows#3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit to dscho/git that referenced this issue Sep 7, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes git-for-windows#3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member

dscho commented Sep 7, 2021

@mikekaganski do you think you could try that?

Yes, I can try to build, but I need help to get an exact command/sequence of actions that would trigger the GC during the pull command.

I opened a PR, and then used the PR branch to kick off a build the Git artifacts (installer and portable Git, for x86_64): https://github.com/git-for-windows/git/actions/runs/1209162450. Once that run succeeds (knock on wood), it will have the installer and the portable Git as a build artifact for you to test.

If you want to try other things, just fetch the PR branch, make your own modifications, push it to your fork on GitHub, then kick off your own git-artifacts build (see https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/ for details how to trigger GitHub workflows manually).

@mikekaganski
Copy link

@mikekaganski do you think you could try that?

Yes, I can try to build, but I need help to get an exact command/sequence of actions that would trigger the GC during the pull command.

I opened a PR, and then used the PR branch to kick off a build the Git artifacts (installer and portable Git, for x86_64): https://github.com/git-for-windows/git/actions/runs/1209162450. Once that run succeeds (knock on wood), it will have the installer and the portable Git as a build artifact for you to test.

Thank you!
I only wanted to know if there is something simple that I can make in my local git repo, so that the next time I run git pull there, I know that GC will start, so that it's possible to see immediately if things change or not (GC succeeds with patch, show errors without, repeatable). Sorry, I'm not an experienced git user :)

@dscho
Copy link
Member

dscho commented Sep 7, 2021

@mikekaganski hmm. I am not quite sure. You could try to create tons of loose objects, but that might not trigger any repack. Or you could potentially lower the threshold for an auto-gc (see here for details).

@dscho
Copy link
Member

dscho commented Sep 7, 2021

@mikekaganski here is the artifact: https://github.com/git-for-windows/git/suites/3701256089/artifacts/90330039

Please note that the CI build suggests that there is a bug with respect to the commit graph; If you have enabled that feature, please do not use that build, otherwise I would be glad if you tested it.

@mikekaganski
Copy link

Thanks - but in the meantime I have built it myself, and am testing ATM, with the gc.autoPackLimit set to 1. It initiated gc on pull; I will re-build without the change, and re-test, to see if there's a difference

dscho added a commit to dscho/git that referenced this issue Sep 7, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes git-for-windows#3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@mikekaganski
Copy link

At last I can report the "comparative" results (if a single test may be deemed such :) )

I have built git with your change; pulling using that version luckily triggered GC immediately; and it passed flawlessly.
After that, I restored the builtin/pull.c file, built git, and did my usual stuff (pulling, fetching, checking out, etc.), until today it finally started gc at pulling. The gc produced the usual "Unlink of file '.git/objects/pack/pack-32a943213f187b05c5470d34ceb619466da1e158.idx' failed. Should I try again? (y/n)" message.

So based on this limited testing, I would assume the change works as intended - thank you! Looking forward to see it implemented :)

@dscho
Copy link
Member

dscho commented Sep 8, 2021

Yeah!

@mikekaganski
Copy link

Just a note from an outsider, who admittedly doesn't know much about the low-level implementation details.

It looks like this is a long-lasting problem. Hopefully this change fixes the last piece of it ;) - but possibly it could be also changed using a different sharing mode when opening files on Windows - see https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea, and its dwShareMode argument, allowing to pass a set of FILE_SHARE_DELETE, FILE_SHARE_READ, and FILE_SHARE_WRITE. The first of them feels to do what is needed here - but again, I may be completely wrong; please excuse for the noise if so :)

@dscho
Copy link
Member

dscho commented Sep 8, 2021

Right, there have been discussions about this. I forgot the details but from my vague memory seem to recall that it was not possible to use those flags (was it the fact that you can delete the file but then not recreate it under the same name until you reboot? Something like that).

BTW there is now a new snapshot which has the fix. Since its executables are code-signed just like official releases (which anti-malware likes a lot more than unsigned binaries), you might want to switch to that snapshot version for the time being.

@mikekaganski
Copy link

Thank you for the explanation! Yes, this makes perfect sense - I didn't think about re-creating; that is a known problem of the scenario.

BTW there is now a new snapshot which has the fix.

Thank you so much! Already using it! :-) Have a nice day, and thanks again!

gitster pushed a commit to git/git that referenced this issue Sep 8, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes git-for-windows#3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dscho added a commit to dscho/git that referenced this issue Oct 13, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes git-for-windows#3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 13, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Oct 14, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 25, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Oct 26, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 28, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 28, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 28, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 28, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 28, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
git-for-windows-ci pushed a commit that referenced this issue Oct 29, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Nov 1, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Nov 4, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Nov 4, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Nov 10, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this issue Nov 16, 2021
On Windows, files cannot be removed nor renamed if there are still
handles held by a process. To remedy that, we try to release all open
handles to any `.pack` file before e.g. repacking (which would want to
remove the original `.pack` file(s) after it is done).

Since the `read_cache_unmerged()` and/or the `get_oid()` call in `git
pull` can cause `.pack` files to be opened, we need to release the open
handles before calling `git fetch`: the latter process might want to
spawn an auto-gc, which in turn might want to repack the objects.

This commit is similar in spirit to 5bdece0 (gc/repack: release
packs when needed, 2018-12-15).

This fixes #3336.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@byblinkdagger
Copy link

Is there any progress to update? I met this error on a company computer(Everything workes well on other computers).It took me 2 days to try to fix this but failed.

git version 2.36.0-rcl/2.35.2/2.35.1/2.28.0

Microsoft Windows Professional [Version 10.0.19044.1288]

git clone : Unlink of file 'E:/myProject/.git/objects/pack/pack-c9637f8851d29a019d4b2ab4e162bd661888bcf7.keep'

git checkout : Error:unable to unlink old myProject/app/src/main/xxx.file': Permission denied(xxx.file is the same file on different branch,it seems [Git] cannot access these same files.But I confirmed that the permission is normal)

git stash/rool back:Unlink of file '.git/index.stash.9100' failed

I have to write the code on this company computer now and diff/merge them manually on other computer now.

I confirm that there is no process occupancy problem with ProcessExplore.

Can someone help?Thanks very much.

@dscho
Copy link
Member

dscho commented Apr 19, 2022

@byblinkdagger your issue might be a very new bug, but with the same suggestions to use Sysinternals' HANDLE or Process Explorer to investigate further.

@byblinkdagger
Copy link

@byblinkdagger your issue might be a very new bug, but with the same suggestions to use Sysinternals' HANDLE or Process Explorer to investigate further.

Thanks for your reply.I will continue to check with your suggestions in my spare time.In fact, when I cloned the Git for Windows SDK,i met the same error.

Cloning the Git for Windows SDK...
warning: templates not found in E:/gitsdk/mini/mingw64/share/git-core/templates
Initialized empty Git repository in E:/gitsdk/.git/
remote: Enumerating objects: 135304, done.
remote: Counting objects: 100% (135304/135304), done.
remote: Compressing objects: 100% (84762/84762), done.
fatal: index-pack failed
Unlink of file 'E:/gitsdk/.git/shallow.lock' failed. Should I try again? (y/n)

Fortunately, when I work alone(use a personal branch,and just use git commit/push command,), it seems normal.I suspect it may be related to the company's internal control and monitor software now(only on my computer,so sad :) ).
If I have new findings, I will give feedback in time.

@byblinkdagger
Copy link

What Process Explorer show :
git_error

Everything seems ok?(Sorry,i lack knowledge of windows and git.)

@dscho
Copy link
Member

dscho commented Apr 21, 2022

It is quite possible that some part of Git tries to delete the .pack/.keep files while another part still holds on to an open handle.

Unfortunately, these things are notoriously hard to diagnose. Do you have any simple way to reproduce the issue reliably?

@byblinkdagger
Copy link

Do you have any simple way to reproduce the issue reliably?

It only happens on my company computer(Ghost Windows version ).Everything is normal on my colleague's computer.Now i use VMware(create a vm) to deal with team work due to urgent project schedule.

@dscho
Copy link
Member

dscho commented Apr 26, 2022

It only happens on my company computer(Ghost Windows version ).

That smells like anti-malware running a bit out of control.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants