Broken pipe due to fast-import crash #4
Comments
|
The only way I could see this happening is if the mark files somehow get corrupted, which can happen if you get a crash while pushing. This should be fixed if you use the latest Git (2.0), and the latest git-remote-hg (not the original push crash, but the subsequent marks sync crashes). You can check if your mark files are correctly sync'ed with a tool I wrote: https://gist.github.com/felipec/10551806 I can run it like: |
|
I'm running git v2.0.0 and since then have been regularly downloading git-remote-hg from this repo as instructed in the readme. The latest git-remote-hg still reports the same issue. I also ran git-marks-check with the following output: Is there any chance to re-generate or fix the mark files without re-cloning the repo? |
That will prevent future desynchronizations, it won't help with the already screwed up marks. You can try doing this: That will remove the marks that are missing from one side. In theory they will be synchronized the next time you fetch. |
|
Not quite there. Running that command resulted in git -c diff.mnemonicprefix=false -c core.quotepath=false -c credential.helper=sourcetree fetch origin
Traceback (most recent call last):
File "/usr/local/bin/git-remote-hg", line 1322, in <module>
sys.exit(main(sys.argv))
File "/usr/local/bin/git-remote-hg", line 1306, in main
do_import(parser)
File "/usr/local/bin/git-remote-hg", line 721, in do_import
export_branch(repo, branch)
File "/usr/local/bin/git-remote-hg", line 613, in export_branch
export_ref(repo, branch, 'branches', head)
File "/usr/local/bin/git-remote-hg", line 557, in export_ref
print "from :%s" % (rev_to_mark(parents[0]))
File "/usr/local/bin/git-remote-hg", line 440, in rev_to_mark
return marks.from_rev(rev.hex())
File "/usr/local/bin/git-remote-hg", line 173, in from_rev
return self.marks[rev]
KeyError: '817d07c976851546c103f8d8874cd5b3d2a5b4e2'
fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_33170
fatal: Error while running fast-import
Completed with errors, see aboveWith the corresponding fast_import_crash_33170: |
|
Do you have that commit in the repository? (817d07c976851546c103f8d8874cd5b3d2a5b4e2), and is that commit in the 'marks-hg' file? |
|
Actually, maybe you need to remove the contents of the "tips" section in the 'marks-hg' file. |
|
That commit is not in the git repo or the marks-git file. Why are there two sets of marks files (under .git/hg/hg/ and under .git/hg/origin/)? Removing the contents of tips from both marks-hg files and then trying to fetch yielded: git -c diff.mnemonicprefix=false -c core.quotepath=false -c credential.helper=sourcetree fetch origin
fatal: mark :26951 not declared
fast-import: dumping crash report to .git/fast_import_crash_34635
fatal: Error while running fast-import
progress revision 2799 'default' (2800/2819)
Traceback (most recent call last):
File "/usr/local/bin/git-remote-hg", line 1322, in <module>
sys.exit(main(sys.argv))
File "/usr/local/bin/git-remote-hg", line 1306, in main
do_import(parser)
File "/usr/local/bin/git-remote-hg", line 721, in do_import
export_branch(repo, branch)
File "/usr/local/bin/git-remote-hg", line 613, in export_branch
export_ref(repo, branch, 'branches', head)
File "/usr/local/bin/git-remote-hg", line 596, in export_ref
print msg
IOError: [Errno 32] Broken pipe
Completed with errors, see aboveThe corresponding output from fast_import_crash_34635 is: Running git-marks-check after this reports a bunch of missing marks. |
Not in the Git repo, in the Mercurial repo, and not in the marks-git file, in the marks-hg file. And in order to fix it you should both run |
|
My bad. It's both in the hg repo and the marks-hg file, but only within the tips dictionary. default and bookmarks/master point to it. I ran |
|
FYI: The content of tips don't get regenerated for either of the marks-hg files. Is .git/hg/hg/ a deprecated location of the marks files? |
|
Actually, the error you get about mark 26951 seems to be related to notes, so you probably need to remove the section 'last-note' from the marks-hg file.
No, that's the location of the hg information about the "hg" remote. Do you have a remote named "hg"? If not, maybe you did in the past and you can safely remove that. |
|
That did it! I ran
I once renamed the 'origin' remote 'hg', which explains the out-of-date contents of those marks files, but I've since reverted back to naming it 'origin'. I seem to remember that it wouldn't push to the hg repo when the remote was named anything but 'origin', but I can be mistaken. Thanks for the prompt response and for sticking around until this got resolved! |
|
Cool. This synchronization issue shouldn't happen again if you are using Git v2.0 and a recent git-remote-hg. So I'll close this as resolved, reopen if it happens again. |
|
Well, today it happened again, but thankfully the same fix worked. I'm beginning to understand what might be triggering the issue, so there's a chance you might be able to reproduce it. My setup is as follows: I have a copy of the mercurial repo stored locally, from which I cloned the git repo. My project enforces a workflow where all commits have to be rebased instead of merged, so when I was ready to push a commit today, I:
For some reason, the commit appeared on the hg repo orphaned from the history. I proceeded to strip the commit from the hg repo, after which I could no longer fetch from git. The cause seems to be simple. The marks-hg file's 'last-note' was now pointing to a commit that wasn't there. I ran At this point, I'm not sure whether Hopefully you'll be able to reproduce the error and tweak git-remote-hg to handle this scenario. I wish I didn't have to constantly rebase, but that's the name of the game here. Let me know if I can be of help. Also, It wouldn't let me re-open the issue, so I'll let you handle that however you see fit. |
|
@felipec: Were you able to reproduce the issue? |
|
This issue reappeared today. Here's a succinct summary of a way to resolve it for future reference:
|
|
I just ran into this issue and the steps provided by @fgb did the trick. Thanks! |
|
I'm starting to hit this issue constantly. I'm almost ready to set up an alias that runs the above steps before every fetch. :-/ |
|
|
|
@tomxtobin: that sounds a lot worse than what I've been experiencing. I bump into this issue when fetching from the mercurial repo and then proceeding to alter its history. The next fetch always fails with the broken pipe issue because the last commit git pulled from hg is no longer there (i.e. the tip's hash has changed). |
This doesn't solve the actual issues, but helps mitigate them in many cases. I am not completely sure what the "actual" issues are, but one seems to be that it is apparently relatively easy to end up with a corrupt marks file (and indeed, the code loading / saving marks does not attempt to be particular robust). The other problem hinted at in issue #4 seems to be when tips in the remote hg repository change, then this seems to also cause troubles. See also: felipec#4 felipec#10
|
Has anyone actually managed to create a reproducible test case? Spent a few hours in trying to hunt the cause, but failed. The marks-git file was missing marks which appeared in marks-hg after the crash. Could it be related to closed branches and merges? What I did find was a corrupted hg repo. That is, when running The first crash report (after Further git fetches will always result in this crash: |
This doesn't solve the actual issues, but helps mitigate them in many cases. I am not completely sure what the "actual" issues are, but one seems to be that it is apparently relatively easy to end up with a corrupt marks file (and indeed, the code loading / saving marks does not attempt to be particular robust). The other problem hinted at in issue #4 seems to be when tips in the remote hg repository change, then this seems to also cause troubles. See also: felipec#4 felipec#10
|
@Lekensteyn: Maybe a closed branch or a merge can trigger it, but it hasn't been my experience. I rarely close a branch in hg and have merged branches in git and hg and pulled / pushed them across git-remote-hg without issues. |
|
@tomxtobin: Are you still experiencing the crippling broken pipe on every fetch? |
|
@fgb Yes, nothing's changed there, unfortunately. |
|
We need to create a script that reproduces this issue. |
That is strange. We'll need to find a way to reproduce that, but that's separate from the issue here.
That is not so simple. A lot of things need to go wrong for the last-note to not be there. And that is the issue that is breaking things for you. I've found a way to reproduce this, but again, that's separate from the original bug. I've opened a new issue (#56). |
|
As a simple workaround, it seems deleting the mark files and rm .git/hg/origin/marks-* |
|
@eregon Thanks for your tip! |
|
@eregon Great tip! It does take a bit to regenerate the mark files, but it seems to be a great way to fix/reset git-remote-hg in the worst case scenario short of re-cloning the hg repo. |
Optionally don't mangle git usernames with quotes
Sporadically, I get a broken pipe error when fast-import crashes, which prevents syncing the hg repo.
Here's the relevant output:
The last time this happened, I resolved it by re-cloning the hg repo, which is an extreme solution. I'd rather fix this differently because I keep multiple clones of the git repo that all need to be re-cloned if I alter the original.
Here's the related fast_import_crash_15105:
Any help would be welcome. Thanks for the great plugin.
The text was updated successfully, but these errors were encountered: