How to proceed with interrupted svn2git

[gundalar]

Hi,

I thought this is a pretty common scenario, but somehow couldn't find any useful suggestion on how exactly to proceed. So please excuse me if it is already discussed elsewhere.

I am converting a fairly large svn repo , something like 70 GB by using svn2git ..repo-url in verbose mode. After it ran for over 2 days and still not done with git svn fetch, I lost network connection; so the process got interrupted.

I ran the git svn fetch manually so that it doesn't run the git init again. I thought it would be smart enough to pick up from where it left but going by the messages looks like it is re-fetching the same content. Appreciate if someone can point me how to proceed assuming I may not be continuously on the network. If svn2git can be broken down into git init, git fetch etc, what will be the subsequent commands.

If it helps I migrated the trunk alone with --trunk trunk --nobranches --notags, which itself took like 6 hrs or so. Now I want to migrate the branches + tags

Thanks.

[kayhadrin]

Hi Ramakrishna,

I've also done a large svn2git import albeit from a smaller svn repo (21GB) and had this kind of issue too.
My workaround was to make a svnsync script to create a local svn mirror for svn2git to use.

So basically, here's what I've done:

• create a script "svn_mirror" to make a svn mirror repo of the remote svn repo. I wrapped the svnsync call in a loop so that it can resume the synchronisation in case of a network issue.
• create a script "svn2git_import" to run svn2git in a loop that repeat itself every minute.
• run svn_mirror first
• start a local svn server on the repository created from svn_mirror
• run svn2git_import using the local svn server that I just launched

The general idea is that:

• While svn_mirror runs, svn2git_import converts whatever it can get from the local svn repo.
• If, by any chance, the remote svn server is not available, it won't break the local import because svnsync will still maintain the integrity of the local svn repo.
• The svn2git import is normally not fast enough to catch up with the svn sync, so it should not run out of data until the end of the mirrorring process.
• However, if you have people continuously committing in svn during the import process, svn2git will be able to resume the conversion.

AFAIK, svn2git is designed to be able to resume from a git repo created with it. It creates custom import filters during the repo initialisation, so using "git svn fetch" manually is probably not correct.
Try running svn2git (or the svn2git_import script) and interrupt at the first time when you see that it starts to do a fetch. You can then run it again and svn2git will be able to resume the fetch.

Lastly, depending on the number of svn merges that you have in your svn repo, git takes more and more time to resolve the svn merge information. So be prepared to see very long processing times for each merge commit.

I hope this helps.

Cheers,

David

[gundalar]

Hi David,

Thanks for the suggestion. I tried to follow similar approach.

I used svnadmin dump to create a mirror of the svn repo, which itself was quite fast ( couple of hours). Then I locally ran the svn2git task on the mirror so that there wouldn't be any network issues. It ran for 1.5 days and then stopped responding. I guess it must have been memory issues as I later came to know that the RAM provisioned was only 1GB. It got most of the stuff (say 80%) except for the branches I wanted. I re-ran the task but it was not getting the branches I actually wanted to migrate.

To make things easier, I only had to migrate 3 branches so I ended-up migrating branch by branch. To do this I had to find the branching point in the git tree going by the commit message in the svn graph when the branch was created from the trunk.

Thinking about it, had I opted out the large amount of documentation from migration or had more RAM been allocated, it would have probably completed the original task but the bottom line is re-running the original svn2git task once it got interrupted or stopped responding didn't help.

Thanks,
Ramakrishna

[bluesquall]

Had a similar problem but lost the connection when my VPN dropped out. I'm trying again using a machine on the same network first, but I'll look into your local mirror approach if it hasn't completed by tomorrow.

[bluesquall]

With svn2git version 2 or later, you may also want to try using svn2git --rebase as suggested here. This is the same thing you would do to keep the git mirror synced with the svn source, as documented in the README.