very very slow interactive rebases

[fommil]

This appears to be a common problem:

• http://stackoverflow.com/questions/2835775
• http://stackoverflow.com/questions/4485059

and I've tried everything (that I'm allowed to do... I don't have Administrator access and will never get it). I'm using Windows 7 64 bit and have no control over changing that.

All the fs cache options mean that a git status is relatively fast (maybe a few seconds) on a humongous codebase, but still nothing compared to what I'm used to on GNU/Linux. Anything like an interactive rebase (even to squash two trivial commits) can take as long as a minute.

Looking at the Task Manager, I can see lots and lots of forked git subprocesses. This may be the cause of the performance problems: I understand this is very slow in Windows.

Would it be a major piece of work to redesign the subprocess execution to be in-process? Even if it meant doing everything in serial.

I'm considering alternatives, possibly even contributing to http://www.eclipse.org/jgit/ as an alternative, as it does not require any subprocess forking. I suspect that the performance cost of the JVM startup (and loss of C-level optimisations) is insignificant compared to the process forking overhead on Windows.

[dscho]

This appears to be a common problem:

Yes, it is known, and there was already somebody working on it, but he decided that he ran out of time: #461

and I've tried everything (that I'm allowed to do... I don't have Administrator access and will never get it).

Nope, you did not try to rewrite the shell script that implements interactive rebase in C.

[fommil]

😄 I should rephrase that as "I tried everything sane". But I'm very happy to hear that insane people exist. I'd be willing to fund this work if it is small enough.

[dscho]

I'd be willing to fund this work if it is small enough.

It is not small.

Just for comparison: the latest Google Summer of Code sponsored the shell script -> C conversion for git pull and git am, i.e. two shell scripts. Google paid the student $4,500 in total IIRC. The rebase functionality consists of four shell scripts.

[fommil]

@dscho good to get some numbers on it!

BTW, there is a new GSOC coming up, is there any interest from capable students for the final four scripts? I'd be willing to fund one, maybe two, if there is a shortfall.

[fommil]

this has been marked as an upstream matter. Is there a ticket upstream that I can subscribe to?

[dscho]

Is there a ticket upstream that I can subscribe to?

No.

[fommil]

@dscho I'm investigating funding possibilities. Would you be able to find somebody who would be willing to work on this?

[fommil]

what about perl instead of native? Might be a lot easier to do that first and get some quick / easy wins.

[dscho]

what about perl instead of native? Might be a lot easier to do that first and get some quick / easy wins.

Wishful thinking! Our Perl is using a POSIX emulation layer, so it is not a speed demon. And you would still have to call Git pretty much as many times as the shell scripts do, and those spawned processes are what kills performance.

[pyokagan]

Hi @dscho, @fommil,

I'm planning on participating in GSoC again this year. I'm interested in tackling this challenge if Git applies as well :-)

For now, I'm working on a prototype to see if this will be feasible during the GSoC timeframe. I'll send a WIP PR to this repo when I get the basic structure done.

Thanks.

[fommil]

(Watching with excitement)