"Git is already running" on Windows

[dabrahams]

Most commands appear to get "stuck" in this state. I can `$' and then kill the buffer without even a prompt. Very strange.

[YorkZ]

I'm having this problem all the times on Windows 7 at home (i7 processor, 8G RAM), but never had it on Windows XP at work (old computer with 1G RAM). The problem is that the previous git process is still alive, and I have to kill the process manually each time I got into this problem.

[tarsius]

Is this still an issue? If so do you have any insight into why that might be so?

[razcampagne]

I used to have the exact same problem but when I updated via elpa yesterday, it worked like a charm.

[tarsius]

I don't think this problem is Windows specific. This happens when git gets killed at the wrong time for whatever reason. This could be a C-g or git crashing. The latter possibly happens more often on Windows than *nix.

Regardless we need to make sure that magit does not get in a state were it refuses to call git because it thinks that is still running. Pull request #676 seems to address this, but unfortunately I did not find the time to inspect it yet.

[razcampagne]

This may also not be related to magit because when this problem start occuring, if I just kill magit-process buffer and continue working, it will fail to detect git was killed nearly every time I use a magit command. And from that moment, emacs itself will be a bit lagging (cursor movement will not be as smooth as normal) and gnus will also become slow.

It may just be a bug in emacs on windows.

[dabrahams]

But it happens on Mac, too

Sent from my moss-covered three-handled family gradunza

On Jul 11, 2013, at 6:32 PM, razcampagne notifications@github.com wrote:

This may also not be related to magit because when this problem start occuring, if I just kill magit-process buffer and continue working, it will fail to detect git was killed nearly every time I use a magit command. And from that moment, emacs itself will be a bit lagging (cursor movement will not be as smooth as normal) and gnus will also become slow.

It may just be a bug in emacs on windows.

—
Reply to this email directly or view it on GitHub.

[tarsius]

I have fixed this with 5633708.

[RadicalZephyr]

FYI, I'm still getting this problem with magit version 20130724.1937. Not positive if that's the latest one, but it seems to be more recent than the last comment on this issue...

Also, the previously mentioned workaround of killing the git-process buffer doesn't work. For me that just causes magit to get well and stuck. It still says that "Git is already running" on every command, and in addition if I try and see the git-process buffer with "$" then it says "No Git commands have run."

Edit: It appears that killing the git-process buffer via kill-buffer doesn't actually end the git process. Doing that manually by executing kill-process did the trick of restoring magit to working status.

I'm on Windows 7 64-bit, with emacs 24.3.1.

[YorkZ]

This is definitely a HUGE problem with Windows 7 64 bits, but not a problem with Windows XP. I have been suffering from this for at least a year and am pretty sure that this a a problem with Emacs running on Windows 7 64 bits because not only does Magit has the issue but also the "compile" has exactly the same issue when compiling a C++ project.

I have been using mwolson's patch for several months now and it had helped tremendously. Previously both "compile" and Magit gave me the headache but now only the "compile" is giving me headache. Mwolson's patch is something I can't live without and thank you very much mwsolson.

Whenever the problem shows up, it is exactly as what razcampagne had mentioned, "cursor movement will not be as smooth as normal".

I haven't tried "5633708" yet but I'm wondering whether "5633708" had killed the git process which is the most important. If the git process is not killed the "fix" might just mask the symptom without addressing the root cause.

This problem becomes really bad if you are using the `speck-mode' as well.

[YorkZ]

Hi mwolson,

Your patch worked pretty well for me in the past a few months. Recently I pulled the latest magit and it looks like they have changed a lot making it a bit harder to merge your fixes. For this reason I have temporarily dropped you patch and switched back to the latest magit. Unfortunately I'm having the "git was already runing" problem all the time again. It is really unfortunate that "they" didn't want to merge you fixes. I would appreciate it if you could take some time to merge your fixes into the latest magit.

Thanks in advance

[tarsius]

Apparently github doesn't understand what "This might, or might not, fix #487." means :-)

[tarsius]

The relevant paragraph from 83fb910 reads:

This might, or might not, fix #487.  (I am just guessing, I don't
have Windows and until someone who does, actually investigates the
root cause of that issue, guessing is all I can do.  Let's hope I
am guessing right.)

I actually have only little hope that this fixes anything related to this issue. I have before suggested setting magit-process-connection-type to nil and it did not help.

On the other hand I have made quite a few changes to the process handling, so maybe I have fixed this issue by chance. However it is more likely that it just appears like I did: Magit now supports multiple parallel processes, so if the issue actually is that some process never finishes (which I am guessing is the issue) that will no longer prevent new process from being started.

That's an improvement I suppose, but it would still be nice to know whether that is the issue and why that happens.

While refactoring the process code I came across some dubious code in magit-process-filter:

      ;; Find last ^M in string.  If one was found, ignore everything
      ;; before it and delete the current line.
      (let ((ret-pos (length string)))
        (while (and (>= (setq ret-pos (1- ret-pos)) 0)
                    (/= ?\r (aref string ret-pos))))
        (cond ((>= ret-pos 0)
               (goto-char (line-beginning-position))
               (delete-region (point) (line-end-position))
               (insert-and-inherit (substring string (+ ret-pos 1))))
              (t
               (insert-and-inherit string))))

That was added five years ago in d1f0df3 and makes sure we see:

Do stuff 1 of 100

instead of

Do stuff 1 of 100
... <98 lines>
Do stuff 100 of 100

But due to differences in eol encoding between Windows (CR+LF) on one side and pretty much every other popular contemporary OS (LF) that could be the problem. The above code filters CR. See http://en.wikipedia.org/wiki/Newline.

Could you please open the *magit-process* buffer and verify that the heading of each process section eventually gets prefixed with the exit code of the respective process (the green or red number). If that doesn't happen, that means that the process never finished. If you find such a process please post the command line along with the output here. Also post the output from running the same thing in a shell.

[npostavs]

@YorkZ wrote:

Whenever the problem shows up

Do you know how to trigger the problem? I just tried out magit on a Windows 7 64bit box, and everything seemed to work fine (both latest (b55fcf5) and before merge of pr/process (bbb5f78)). But I gather the problem doesn't show immediately?

[tarsius]

No I don't (Edit: ah you weren't asking me, well I don't know :-). One problem so far was insufficient maintainer-affected_user alignment, e.g. by the time some maintainer was ready to look into this issue, the affected user had given up :-/ In other words I was never able to get any reliable information, but that is partly my fault. So let's hope you run into this problem, giving us another change to finally fix this :-)

[npostavs]

(Edit: ah you weren't asking me, well I don't know :-)

Well, I was sort of directing the question to @YorkZ, but of course anyone who has some insight into this could answer...

I found this mentioned in etc/PROBLEMS:

** Antivirus software interacts badly with the MS-Windows version of Emacs.

The usual manifestation of these problems is that subprocesses don't
work or even wedge the entire system. In particular, "M-x shell RET"
was reported to fail to work. But other commands also sometimes don't
work when an antivirus package is installed.

The solution is to switch the antivirus software to a less aggressive
mode (e.g., disable the ``auto-protect'' feature), or even uninstall
or disable it entirely.

This wouldn't be specific to Windows 7 64bit, but perhaps that isn't the relevant difference: it may affect those machines only because they are newer with newer and more aggressive anti virus software installed.

[YorkZ]

@npostavs: how long have you been using Magit in Windows box? I think you might
need to use it longer. I used to get the problem at least 20 times a day. Also
how did you start Emacs, did you run "runemacs.exe" from Windows shortcut? I
used to run emacs by clicking shortcut to "runemacs.exe" but I don't anymore,
now I run emacs by executing "emacs.exe" from msysGit bash and the problem had
dramatically improved. In fact, running emacs from msysGit bash not only
improves this problem but also, improves a bunch of other problems, have no idea
why.

But running emacs.exe from msysGit bash doesn't fix the problem, it still
happens, averagely 3 to 5 times a day now.

With regard to Antivirus, I guess that might make the issue worse. I have
Antivirus on my work computer but not on my home computer, and I suffer from the
issue both at work and at home.

@tarsius: thank you very much for your effort to address this issue,
unfortunately I can't try your new commits as a new problem (issue #1134) is
holding me back. I will appreciate it if you could solve 1134 so that I will be able
to keep up with you new improvements. Your have done really amazing work on
Magit, thanks again.

York

[npostavs]

how long have you been using Magit in Windows box? I think you might
need to use it longer. I used to get the problem at least 20 times a day.

But there's no specific thing that triggers it? eg: it happens more often during pushing or something like that? I haven't been using it too long on this machine, but no problems so far. I'm just a bit worried that my usage pattern will be a bit different from yours, and somehow I won't hit the bug.

Also how did you start Emacs, did you run "runemacs.exe" from Windows shortcut?

Yes, exactly like that.

[YorkZ]

@npostavs: The problem happened frequently after committing, rebasing and some
times pushing as well. The good news is that the problem hasn't hitted me yet
since using the latest Magit a few hours ago, too early to say anything though,
but we'll see. Thank you npostavs and happy new year.

[tarsius]

As per #487 (comment), please try this patch:

diff --git a/magit.el b/magit.el
index 8f73e56..30d4a09 100644
--- a/magit.el
+++ b/magit.el
@@ -3311,17 +3311,7 @@ repository are reverted using `auto-revert-buffers'."
       (setq string (propertize string 'invisible
                                (magit-section-hidden
                                 (process-get proc 'section))))
-      ;; Find last ^M in string.  If one was found, ignore everything
-      ;; before it and delete the current line.
-      (let ((ret-pos (length string)))
-        (while (and (>= (setq ret-pos (1- ret-pos)) 0)
-                    (/= ?\r (aref string ret-pos))))
-        (cond ((>= ret-pos 0)
-               (goto-char (line-beginning-position))
-               (delete-region (point) (line-end-position))
-               (insert-and-inherit (substring string (+ ret-pos 1))))
-              (t
-               (insert-and-inherit string))))
+      (insert-and-inherit string)
       (set-marker (process-mark proc) (point)))))

 (defun magit-process-logfile-filter (process string)

... and report whether that makes these errors go away on Windows.

I might apply this patch even if it doesn't help, because even on systems that use LF for newlines a file might actually contain CR (for some other reason) and the above code would corrupt that. Having a long list of 1 of n ... n of n in *magit-process* is a minor inconvenience in comparison (do we actually ever run a command that produces such output?).

[YorkZ]

@tarsius: Just tried the above patch and it didn't help with staging individual hunks.

[YorkZ]

Back to the origin problem, the problem just hitted me right now. First, here is
the output of the list-processes:

edit-server     listen   *edit-server*  --           (network server on local)
git             run     *magit-process* --           git --no-pager apply --cached -
server          listen  --              --           (network server on local)

Then goes the contents of "magit-process" buffer (the prefix "0" are
green):

1 git --no-pager apply --cached -
error: patch failed: misc-utils.el:1
error: misc-utils.el: patch does not apply


0 git --no-pager checkout -- misc-utils.el


0 git --no-pager add -u misc-utils.el


0 git --no-pager commit --
Waiting for Emacs...
[master c84e900] Unix line ending
1 file changed, 1603 insertions(+), 1603 deletions(-)


0 git --no-pager apply --cached -


0 git --no-pager reset -q HEAD -- misc-utils.el


0 git --no-pager apply --cached -


0 git --no-pager reset -q HEAD -- misc-utils.el


git --no-pager apply --cached -
<stdin>:10: trailing whitespace.
(require 'highlight-symbol) 
warning: 1 line adds whitespace errors.


git --no-pager apply --cached -
<stdin>:10: trailing whitespace.
(require 'highlight-symbol) 
error: patch failed: misc-utils.el:17
error: misc-utils.el: patch does not apply


0 git --no-pager reset -q HEAD -- misc-utils.el

Hope this helps.

[tarsius]

Just tried the above patch and it didn't help with staging individual hunks.

And on second thought, it wouldn't make much sense if it did. The patch affects output from git. And for a valid patch not to apply git's input would have to be somehow corrupted.

But generally I think we are on the right track looking at encodings of line endings.

Please experiment with core.eol, core.safecrlf, and core.autocrlf.

While eol problems might explain why a patch doesn't apply, it seems strange that such problems would keep git processes from ever finishing, as appears to be the case.

[tarsius]

Also git applys --ignore-space-change, --ignore-whitespace, and --whitespace=<action> options.

[npostavs]

While eol problems might explain why a patch doesn't apply, it seems strange that such problems would keep git processes from ever finishing, as appears to be the case.

The git not finishing part is probably this issue (I haven't managed to reproduce that), but the patch not applying is something else, it happens on linux too, see #1139 (comment).

[tarsius]

I fixed the newline problem by using the --ignore-space-change option mentioned above.

[YorkZ]

I don't think this issue had ever been addressed. I'm still suffering from it every day, no exception.

[npostavs]

There can't be any progress on this until we figure out what about your setup causes it. Although it's probably not really a magit issue since you said:

the "compile" has exactly the same issue when compiling a C++ project.

[YorkZ]

I'm pretty sure the root cause of this problem is in NTEmacs itself, because Magit is not the only one that suffers this issue. However, Magit is the one that suffers the most, and I'm sure it can be improved because pull request #622 used to work pretty well for me. The problem is that pull request is no longer compatible with the current Magit and I don't want to fall back to old Magit either because the current Magit really has huge improvement. I'm wondering if pull request #622 can be considered more seriously to make our life easier.

And I don't think this issue is specific to my setup because obviously I'm not the only person who suffers from this. If you use NTEmacs on Windows 7 I'm sure you will easily run into this issue.

[npostavs]

And I don't think this issue is specific to my setup because obviously I'm not the only person who suffers from this. If you use NTEmacs on Windows 7 I'm sure you will easily run into this issue.

There were a few other reports, so it's less specific than your particular setup, but NTEmacs Windows 7 64 bit isn't specific enough because I have that too and it doesn't happen to me. Also, I searched for previous reports about Emacs subprocesses hanging and the only thing I turned up is the anti virus thing.

[tarsius]

@YorkZ I have installed a change today that tries to use "git" directly instead of one of the Window specific wrappers. If the problem was with these wrappers then that might "fix" this issue. a3c077e

[YorkZ]

@tarsius: I have pulled the latest change, and will report back. Thank you.

@tarsius and npostavs: One thing I think I had mention before, which I believe is the single most important factor, is that I'm using speck.el for spell checking. This program uses idle timer and I suspect that anything utilizing idle timer would make the issue much worse. For this reason, may I suggest that you try out speck.el (with Windows 7)? This program by itself is an excellent spell checking program, much faster (and usable) than flyspell.

[YorkZ]

I'm still having exactly the same issue. Is it possible to use call-process instead of start-process, I noticed that anything that uses start-proces seems to be prone to having this problem.

[vanicat]

Tramp integration need start-process. I believe a lot of us don't want to loose it.

[tarsius]

So does staging and committing, even locally. And then a few things that are not quite as essential. So no, we cannot make everything synchronous.