New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import gets an error then it's stuck #85

Closed
emilianbold opened this Issue Dec 22, 2016 · 6 comments

Comments

Projects
None yet
2 participants
@emilianbold
Contributor

emilianbold commented Dec 22, 2016

Hello,

I'm trying to migrate the large Mercurial repository http://hg.netbeans.org/releases/ to Git incrementally.

I have successfully used hg-fast-export.sh with a sibling repository http://hg.netbeans.org/main-silver.

On this one though I got this error at some point:

error: unable to write sha1 filename .git/objects/pack/pack-1a87fbb507acdbda56d35e4fc6c1dd11a17859b3.pack: No such file or directory
fatal: cannot store pack file
fast-import: dumping crash report to .git/fast_import_crash_80114

I'm also attaching .git/fast_import_crash_80114
fast_import_crash_80114.zip

I don't know how to recover from this error.

Running hg-fast-export.sh on an incrementally updated repo gives me

master: Exporting simple delta revision 177500/178431 with 0/1/0 added/changed/removed files
fatal: mark :177498 not declared
fast-import: dumping crash report to .git/fast_import_crash_80287
master: Exporting simple delta revision 177501/178431 with 0/1/0 added/changed/removed files
Traceback (most recent call last):
File "/Users/apache/work/fast-export/hg-fast-export.py", line 504, in
notes=options.notes,encoding=encoding,fn_encoding=fn_encoding))
File "/Users/apache/work/fast-export/hg-fast-export.py", line 404, in hg2git
sob,brmap,hgtags,encoding,fn_encoding)
File "/Users/apache/work/fast-export/hg-fast-export.py", line 244, in export_commit
export_file_contents(ctx,man,changed,hgtags,fn_encoding)
File "/Users/apache/work/fast-export/hg-fast-export.py", line 141, in export_file_contents
wr(d)
File "/Users/apache/work/fast-export/hg-fast-export.py", line 37, in wr
wr_no_nl(msg)
File "/Users/apache/work/fast-export/hg-fast-export.py", line 34, in wr_no_nl
sys.stdout.write(msg)
IOError: [Errno 32] Broken pipe

I tried using hg-reset.sh -R to reset to an earlier revision and retry. This worked the 1st time but didn't fix the issue so I tried resetting to an even earlier revision but got another error:

$ ../fast-export/hg-reset.sh -R 176481
Using last hg repository "/Users/apache/work/releases"
Traceback (most recent call last):
File "../fast-export/hg-reset.py", line 117, in
stale,changed,unchanged=get_branches(ui,repo,heads_cache,marks_cache,mapping_cache,options.revision+1)
File "../fast-export/hg-reset.py", line 45, in get_branches
del stale[branch]
KeyError: 'master'

I would appreciate any hints on how to continue.

@frej frej added the user-support label Dec 23, 2016

@frej

This comment has been minimized.

Show comment
Hide comment
@frej

frej Dec 25, 2016

Owner

The first error message is git-fast-imprt complaining (while repacking) and then giving up. Check that your are not running out of disk space or having permission problems. After that I guess things are corrupt and I have not any suggestions for recovery except starting over with a clean git repo.

I have successfully converted http://hg.netbeans.org/releases/ using git version 2.11.0, so I don't think this is a problem with fast-export.

Owner

frej commented Dec 25, 2016

The first error message is git-fast-imprt complaining (while repacking) and then giving up. Check that your are not running out of disk space or having permission problems. After that I guess things are corrupt and I have not any suggestions for recovery except starting over with a clean git repo.

I have successfully converted http://hg.netbeans.org/releases/ using git version 2.11.0, so I don't think this is a problem with fast-export.

@emilianbold

This comment has been minimized.

Show comment
Hide comment
@emilianbold

emilianbold Dec 25, 2016

Contributor

I have plenty of disk space and I don't believe there are any permission problems since it's just a local folder...

I'm using git version 2.6.3 and conversion of main-silver took a really long time.

How long did your releases/ conversion take? Did you skip somehow the unnamed heads in releases/?

Anyhow, I'll try again and report back.

Thank you for taking your time on Christmas to reply!

Contributor

emilianbold commented Dec 25, 2016

I have plenty of disk space and I don't believe there are any permission problems since it's just a local folder...

I'm using git version 2.6.3 and conversion of main-silver took a really long time.

How long did your releases/ conversion take? Did you skip somehow the unnamed heads in releases/?

Anyhow, I'll try again and report back.

Thank you for taking your time on Christmas to reply!

@frej

This comment has been minimized.

Show comment
Hide comment
@frej

frej Dec 25, 2016

Owner
Owner

frej commented Dec 25, 2016

@emilianbold

This comment has been minimized.

Show comment
Hide comment
@emilianbold

emilianbold Dec 28, 2016

Contributor

After that I guess things are corrupt and I have not any suggestions for recovery except starting over with a clean git repo.

I would like to avoid this in the future considering conversion takes 48 hours (mine is finishing today).

I assume it's recommended to periodically backup hg2git-heads, hg2git-mapping, hg2git-marks and hg2git-state?

Then I can make another git clone, copy over the hg2git-* files and and resume incremental conversion?

This would also allow me to move conversion to another machine.

Contributor

emilianbold commented Dec 28, 2016

After that I guess things are corrupt and I have not any suggestions for recovery except starting over with a clean git repo.

I would like to avoid this in the future considering conversion takes 48 hours (mine is finishing today).

I assume it's recommended to periodically backup hg2git-heads, hg2git-mapping, hg2git-marks and hg2git-state?

Then I can make another git clone, copy over the hg2git-* files and and resume incremental conversion?

This would also allow me to move conversion to another machine.

@frej

This comment has been minimized.

Show comment
Hide comment
@frej

frej Dec 28, 2016

Owner
Owner

frej commented Dec 28, 2016

@emilianbold

This comment has been minimized.

Show comment
Hide comment
@emilianbold

emilianbold Dec 28, 2016

Contributor

Thank you for your support.

The new conversion finished today without any errors.

With the state files backup I believe we can close this issue.

Contributor

emilianbold commented Dec 28, 2016

Thank you for your support.

The new conversion finished today without any errors.

With the state files backup I believe we can close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment