Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance issues #2104

tarsius opened this issue Jul 21, 2015 · 39 comments

Performance issues #2104

tarsius opened this issue Jul 21, 2015 · 39 comments


Copy link

@tarsius tarsius commented Jul 21, 2015

The performance regression in v2.1 was due to a design flaw in the code that reverted file-visiting buffers when then files changed on disk. On every revert cycle, it was recalculated for each buffer whether it visits a file that is located inside a repository and whether the file tracked. That was to expensive when there were many buffers open and/or the repository tracked many files.

I have now replaced that implementation with magit-auto-revert-mode, a globalized variant of the built-in auto-revert-mode, which does not have this issue.

Originally I opened this issue because after the release many issues about alleged performance regressions were opened and most of these issues appeared to be duplicates. Now that I believe to have addressed the regression, I am closing this.

Magit still has performance issues - mainly the "everything is needlessly recreated all the time" issue. But that's not a regression but a very old design decision. I do intend to address that in v3, see the milestones page and click on "more" below "3.1.0".

The feedback about changes in performance characteristics in v2.1 has been mixed. On one hand many users have reported massive performance gains, but there are also reports of performance having gotten worse.

I intend to address these regressions in v2.5.0, but so far it isn't clear in most cases, what is causing these issues. It seems that depending on repository and usage characteristics performance can still be quite bad. Figuring out what exactly causes these issues, will allow me come up with the appropriate countermeasures.

You can help me with that by accompanying your reports about performance regressions with benchmarks. Instead of reporting that "everything is slow", make an effort narrow it down. Which commands are effected? Is the performance worse in certain repositories - what are the characteristics of those repositories? If you know elisp, then please make an effort to narrow the issue down to certain code yourself.

Also have a look at the existing issues and if one matches the symptoms you are experiencing, then comment on those, providing additional information. #2099, #2097, #2049, #1947.

Other issues about performance which were created after I created this one: #2105, #2108, #2162, #2228.

Copy link

@phil-s phil-s commented Aug 13, 2015

Copy link
Member Author

@tarsius tarsius commented Aug 14, 2015

Thanks @phil-s.

@tarsius tarsius modified the milestones: 2.4.0, later Aug 17, 2015
@tarsius tarsius modified the milestones: later, 3.1.0 Aug 31, 2015
Copy link
Member Author

@tarsius tarsius commented Aug 31, 2015

I have added this to the 3.1.0 milestone, but that doesn't mean that I won't do anything about performance issues until then. But performance issues won't be a thing of the past until diff and log washing is done completely asynchronously and that is scheduled for that release (which might happen before the currently scheduled release date). Also see #2180 for some intermediate performance improvements that are in planning.

@tarsius tarsius mentioned this issue Aug 31, 2015
34 of 34 tasks complete
Copy link

@alamaison alamaison commented Sep 17, 2015

Whatever you've been doing recently has completely fixed the performance issues during merge, for me. Thanks!

Copy link

@ghost ghost commented Oct 12, 2015

emacs 25's alleged support for dynamic library loading and libgit2?

Copy link
Member Author

@tarsius tarsius commented Oct 12, 2015

No, I added a kludge that keeps git from taking a smoke break. I.e. addressed the second bullet point in

Copy link

@alamaison alamaison commented Oct 19, 2015

Alas, the performance issues during merges have returned :( Has something changed back?

I notice that if I C-g while the status buffer is frozen, the buffer seems to display OK, but the diffs aren't colourised. If I move the point to the diff, it becomes colourised.

Copy link

@alamaison alamaison commented Oct 23, 2015

And now its fast again. Whatever you changed, thanks.

Copy link

@dchenbecker dchenbecker commented Oct 24, 2015

One thing I noticed is that now the log display shows all commits to the beginning of time (9600+ commits for the repo I work on daily), whereas before it would show some limited number and prompt for more (I can't remember the exact wording). When I run the log command now the emacs process goes to 100% for 20-30 seconds before showing the log. Right now I have the graph (-g), color (-c) and "show refnames" (-d) options enabled. I'm using git version 2.6.2 (homebrew) and magit version 20151024.220 from melpa. If I limit the number of entries (=n 50) then I get the previous behavior and things are quick. Did the default for =n change from set to nil at some point recently?

Copy link

@johnmastro johnmastro commented Nov 19, 2015

Hi @nshi, how large is the repo, and is it available publicly?

(I probably can't help with the performance issues myself but those are question that I suspect tarsius will ask when he looks at this).

Copy link

@nshi nshi commented Nov 19, 2015

Hi @johnmastro, the repo is not huge, probably medium to large size. You can clone it from

Copy link

@phil-s phil-s commented Nov 20, 2015

nshi: Nearly half the time in that profile is spent on reverting tracked buffers. You can customise Magit's buffer revert options to avoid that (and/or delete as many unnecessary buffers as possible).

Copy link

@fommil fommil commented Nov 20, 2015

@phil-s interesting, I see that being a big slow down in windows too.

Copy link

@phillord phillord commented Nov 23, 2015

I'm currently getting very poor performance on `magit-status'. I have a quite a few changes files (200 or so). Staging one or scrolling is sluggish. I couldn't work out why till I realised the the mode-line counter was showing an enourmous (250k) buffer.

I am guessing that magit inserts the full contents (i.e. diff) into the magit-status buffer the whole time, even when that diff is not being displayed (i.e. is invisible). Unfortunately, with many files this makes for a very large buffer. Or even a few files which are very large.

Copy link
Member Author

@tarsius tarsius commented Nov 23, 2015

The release after the next one will deal with performance. Until then, this FAQ entry applies.

Copy link

@phillord phillord commented Nov 23, 2015

In my case, the commit didn't have that many files, but one of them was enormous (although I didn't know that -- not my file!). This was why I wasn't expecting it. If I was checking in 1000s of files, I'd have seen it coming!

Anyway, if you are already aware of the issue, no worries. Thanks for the software!

Copy link

@rneatherway rneatherway commented Nov 27, 2015

A quick data point. I was working in a branch had about 3000 unpushed commits (rebasing a very old feature branch on latest master) and this made magit-status very slow (on the order of a minute). I tried profiling, but the horizontal truncation kicked in before I got below 85% CPU. Pushing or changing the remote brought performance back to being pretty good.

Copy link

@arunabha arunabha commented Dec 13, 2015

Magit has become noticeably slower for me after upgrading to El-Capitan. Things like staging a file which used to be near instantaneous are now noticeably slower. Here are the details
Computer: MacBook Pro (Retina, 15-inch, Mid 2014), OSX 10.11.1
Magit 20151211.2221, Git 2.4.9 (Apple Git-60), Emacs 24.4.1

I created a test git repo with two files, ran magit-status, then staged and unstaged both files. Elp profile results are below.

magit-process-file                              277         5.0085080000  0.0180812563
magit-run-git                                   6           4.461219      0.7435364999
magit-git-str                                   214         3.9420540000  0.0184208130
magit-rev-parse-safe                            179         3.3028970000  0.0184519385
magit-refresh-buffer                            7           3.08539       0.44077
magit-status-refresh-buffer                     7           3.084037      0.4405767142
magit-refresh                                   6           2.654456      0.4424093333
magit-unstage                                   3           2.438397      0.812799
magit-unstage-1                                 3           2.43826       0.8127533333
magit-stage                                     3           2.0884270000  0.6961423333
magit-stage-untracked                           3           2.088261      0.696087
magit-run-git-no-revert                         3           2.087954      0.6959846666
magit-call-git                                  6           1.806687      0.3011145
magit-toplevel                                  23          1.7027640000  0.0740332173
magit-call-process                              6           1.672918      0.2788196666
magit-git-dir                                   63          1.1386309999  0.0180735079
magit-process-setup                             6           0.9097550000  0.1516258333
magit-mode-get-buffers                          12          0.9011540000  0.0750961666
magit-get-push-branch                           14          0.768957      0.0549255
magit-git-insert                                36          0.6582730000  0.0182853611
magit-process-buffer                            9           0.656204      0.0729115555
magit-process-finish                            6           0.649203      0.1082004999
magit-get                                       35          0.6412009999  0.0183200285
magit-status                                    1           0.540468      0.540468
magit-status-internal                           1           0.540464      0.540464
magit-mode-setup                                1           0.540444      0.540444
magit-get-push-remote                           14          0.5120769999  0.0365769285
magit-process-set-mode-line                     6           0.4647280000  0.0774546666
magit-process-unset-mode-line                   6           0.436554      0.072759
magit-insert-unpushed-to-pushremote             7           0.3880110000  0.0554301428
magit-bisect-in-progress-p                      21          0.3853589999  0.0183504285
magit-git-exit-code                             21          0.3843919999  0.0183043809
magit-insert-unpulled-from-pushremote           7           0.3810400000  0.0544342857
magit-rev-verify                                17          0.322766      0.0189862352
magit-insert-untracked-files                    7           0.261573      0.0373675714
magit-git-success                               14          0.25692       0.0183514285
magit-get-current-branch                        14          0.256704      0.0183359999
magit-git-string                                14          0.256565      0.0183260714
magit-insert-rebase-sequence                    7           0.252101      0.0360144285
magit-rebase-in-progress-p                      7           0.252052      0.0360074285
magit-insert-sequencer-sequence                 7           0.247647      0.0353781428
magit-maybe-save-repository-buffers             13          0.1530109999  0.0117700769
magit-save-repository-buffers                   7           0.152932      0.0218474285
magit-insert-staged-changes                     7           0.13436       0.0191942857
magit-insert-unstaged-changes                   7           0.132929      0.0189898571
magit-git-items                                 7           0.130966      0.0187094285
magit-insert-unpulled-from-upstream             7           0.130334      0.0186191428
magit-insert-bisect-rest                        7           0.129437      0.018491
magit-insert-stashes                            7           0.1293180000  0.018474
magit-insert-merge-log                          7           0.129007      0.0184295714
magit-insert-status-headers                     7           0.128885      0.0184121428
magit-insert-bisect-output                      7           0.128159      0.0183084285
magit-insert-bisect-log                         7           0.127874      0.0182677142
magit-insert-unpushed-to-upstream               7           0.1267070000  0.0181010000
magit-insert-am-sequence                        7           0.125868      0.0179811428
magit-am-in-progress-p                          7           0.125834      0.0179762857
magit-cherry-pick-in-progress-p                 7           0.1245930000  0.0177990000
magit-revert-in-progress-p                      7           0.1229810000  0.0175687142
magit-git-wash                                  14          0.096516      0.006894
magit-mode-get-buffer                           1           0.070613      0.070613
magit-no-commit-p                               3           0.064903      0.0216343333
magit-status-mode                               1           0.018989      0.018989
magit-load-config-extensions                    1           0.017779      0.017779
magit-get-all                                   1           0.017772      0.017772
magit-git-lines                                 1           0.017755      0.017755
magit-wash-sequence                             14          0.002452      0.0001751428
magit-diff-wash-diffs                           5           0.001983      0.0003966
magit-diff-wash-diff                            9           0.00181       0.0002011111
magit--safe-default-directory                   86          0.001738      2.020...e-05
magit-process-git-arguments                     277         0.0012679999  4.577...e-06
magit-expand-git-file-name                      109         0.0011399999  1.045...e-05
magit-diff-insert-file-section                  9           0.0010849999  0.0001205555
magit-section-update-highlight                  16          0.000631      3.94375e-05
magit-process-insert-section                    6           0.00062       0.0001033333
magit-section-ident                             209         0.0005760000  2.755...e-06
magit-diff-wash-hunk                            13          0.00057       4.384...e-05
magit-section-show                              30          0.0004860000  1.620...e-05
magit-insert-heading                            44          0.000438      9.954...e-06
magit-diff-highlight                            11          0.0004099999  3.727...e-05
magit-insert-child-count                        54          0.0003750000  6.944...e-06
magit-display-buffer                            1           0.000371      0.000371
magit-display-buffer-traditional                1           0.000323      0.000323
magit-section-goto-successor                    7           0.0002639999  3.771...e-05
magit-unhighlight-region                        77          0.0002439999  3.168...e-06
magit-cygwin-env-vars                           277         0.0002149999  7.761...e-07
magit-git-repo-p                                3           0.000194      6.466...e-05
magit-mode                                      1           0.000167      0.000167
magit-diff-scope                                29          0.000162      5.586...e-06
magit-delete-line                               54          0.0001549999  2.870...e-06
magit-diff-highlight-recursive                  9           0.000151      1.677...e-05
magit-get-section                               66          0.0001449999  2.196...e-06
magit-section-goto-successor-1                  8           0.000138      1.725e-05
magit-section-match                             11          0.000136      1.236...e-05
magit-file-lines                                7           0.000104      1.485...e-05
magit-revision-set-visibility                   63          9.999...e-05  1.587...e-06
magit-apply--get-selection                      6           9.600...e-05  1.600...e-05
magit-diff-type                                 17          8.8e-05       5.176...e-06
magit-diff-expansion-threshold                  63          8.799...e-05  1.396...e-06
magit-section-hide                              12          7.1e-05       5.916...e-06
magit-current-section                           56          6.099...e-05  1.089...e-06
magit-refresh-get-relative-position             7           5.7e-05       8.142...e-06
magit-diff-highlight-file                       8           5.600...e-05  7.000...e-06
magit-diff-highlight-list                       1           4.8e-05       4.8e-05
magit-pre-command-hook                          37          4.499...e-05  1.216...e-06
magit-region-sections                           26          3.700...e-05  1.423...e-06
magit-process-truncate-log                      9           3.4e-05       3.777...e-06
magit-diff-highlight-heading                    8           3.3e-05       4.125e-06
magit-section-highlight                         5           2.9e-05       5.8e-06
magit-section-make-overlay                      15          2.100...e-05  1.400...e-06
magit-save-window-configuration                 1           1.1e-05       1.1e-05
magit-section-siblings                          11          1e-05         9.090...e-07
magit-tramp-asserts                             1           9e-06         9e-06
magit-section-match-1                           11          8e-06         7.272...e-07
magit-decode-git-path                           9           7e-06         7.777...e-07
magit-wip-commit-after-apply                    6           5.999...e-06  1e-06
magit-wip-commit-before-change                  6           5.999...e-06  1e-06
magit-revert-buffers                            6           4e-06         6.666...e-07
magit-maybe-set-dedicated                       1           1e-06         1e-06
magit-maybe-show-margin                         1           1e-06         1e-06
magit-cancel-section                            9           0             0.0
magit-xref-setup                                1           0.0           0.0

The slowness really takes the joy out of magit :-( . Any ideas ?

Copy link

@adouzzy adouzzy commented Dec 15, 2015

@arunabha I have the same issue. It was significant slow on mac itself than on the linux virtualbox on the same machine. Especially staging files is very slow. This is very annoying.

Copy link

@arunabha arunabha commented Dec 16, 2015

@adouzzy Yes, it's a bummer. I suspect the upgrade to El Capitan might be the culprit as magit was working fine on Yosemite. There is definitely something wonky going on because magit-status takes twice the time as a git status on the command line.

In the meanwhile, I've switched to tig, not a good as magit, but mostly there and lightning fast.

Copy link

@moredhel moredhel commented Dec 16, 2015

I seem to be getting this issue too. I'm trying to track down the culprit now, but am attaching a screenshot of an initial profile in the meantime
screen shot 2015-12-16 at 03 24 23

More Information
This is the average runtime of running epl on both magit-reset and magit-stage-file.

magit-reset       5           7.111757      1.4223514
magit-stage-file  5           5.541849      1.1083698

running through the command-line:

git add Makefile  0.00s user 0.00s system 75% cpu 0.007 total

Early 2015 MBP El Capitan.

Copy link
Member Author

@tarsius tarsius commented Dec 16, 2015

I don't us OS X, so the best I can do is ask Google and then give you third-hand suggestions. It appears the consensus is that you have to reinstall Xcode and then also make sure that you use the Git version you installed by hand instead of the one that comes with Xcode.

Note that just because your terminal is properly configured that does not magically configure Emacs too (unless you start Emacs by typing emacs into said terminal). Check whether M-x magit-version reports the same Git version as git --version.

Copy link

@arunabha arunabha commented Dec 16, 2015

@tarsius Thanks for chiming in.

I've tried setting the git executable explicitly to the more recent git I installed (2.6.4) using magit-git-executable but it didn't change anything.

El Capitan has known dev issues, so I followed instructions here, in particular one of the steps is

xcode-select --install

Is that what you meant ? or did you mean reinstall XCode IDE ? AFAIK, I haven't installed the full XCode IDE, but I can certainly do that if it helps get magit back.

Copy link
Member Author

@tarsius tarsius commented Dec 16, 2015

Yes, I would try xcode-select --install first. I doubt that installing the full XCode IDE would help.

Copy link

@arunabha arunabha commented Dec 16, 2015

Ok, I've already done that and the slow performance continues. FWIW, a git-status on a ~250 file repo takes 0.4 seconds on the commandline and magit-status take 0.9 seconds.

Copy link

@phil-s phil-s commented Dec 16, 2015

Probably worth raising a separate issue for El Capitan performance?

Copy link
Member Author

@tarsius tarsius commented Jan 3, 2016

By default all buffers visiting files inside the current repository are now refreshed, not just those visiting tracked files. This is controlled using the option magit-revert-buffers-only-for-tracked-files, I might still change the default to t before the release.

Please test whether this change results in a performance increase in your case.

Copy link
Member Author

@tarsius tarsius commented Jan 8, 2016

Probably worth raising a separate issue for El Capitan performance?

@phil-s @arunabha @adouzzy @hamhut1066 That might be a good idea, especially because I intend to close this issue soon. (I believe that I have identified the reason for the performance regression in v2 and that #2470 fixes it.) Still hoping that someone will run into the El Capitan issue, finds the cause and fixes it, and then enlightens the rest of us.

By default all buffers visiting files inside the current repository are now refreshed,

I have reverted that because I don't want to be accused of putting users' data at risk. (I believe the additional risk is minuscule, but I had to endure vicious attacks on the matter before.) So you now have to set magit-revert-buffers-only-for-tracked-files to nil yourself. Better yet, use the new implementation from #2470 (if you use Linux). I intend to merge that tomorrow or so.

@alamaison @fommil @nshi I think that #2470 will at least partially address the issue for you.

Copy link

@fommil fommil commented Jan 8, 2016

@tarsius i'm putting together a windows 7 virtual box image for other purposes, but I can't share with you if you're interested. Also, Microsoft offer gratis images for testing purposes now.

Copy link

@fommil fommil commented Jan 8, 2016

(But fundamentally git is slow on windows, so I don't have high hopes)

Copy link

@mgalgs mgalgs commented Jan 11, 2016

We've talked in the past about limiting the number of commits in the log using a revision range rather than --max-count=N. What about doing something like that more generally? I find that a lot of the sluggishness I run into is from section hooks building up a lot of commits (unpulled/unpushed, in particular). It would be nice to have an option like magit-max-revs that would be used to limit the number of commits in the various section hooks as well as the first pass of the magit log buffer.

Copy link
Member Author

@tarsius tarsius commented Jan 11, 2016

@mgalgs That's already possible now. In a log buffer use L - n {number} RET followed by C-x C-s to set and save the limit. By the way, by default -n256 is part of magit-log-arguments.

To be able to do the same in the status buffer you would first have to run this:

(magit-define-popup-option 'magit-log-refresh-popup
  ?n "Limit number of commits" "-n")
@tarsius tarsius modified the milestones: 2.4.0, 3.1.0 Jan 12, 2016
@tarsius tarsius closed this Jan 12, 2016
Copy link

@mgalgs mgalgs commented Jan 15, 2016

@tarsius I was talking about using revision ranges for limiting commits, as discussed in #1990. However, I'm noticing now that this was supposed to be fixed in #2243. For some reason my log buffer and section hooks don't seem to be reaping the benefits :(...

Copy link

@fommil fommil commented Jan 18, 2016

for anybody who doesn't want Emacs to mutate their init.el, this is just a modification of variable magit-log-arguments (and close / reopen magit buffers, because this seems to be cached)

Copy link

@kirk86 kirk86 commented Aug 15, 2017

I can still experience these performance issues when I commit a lot of files using magit. When I do the same using git over cli, I see no issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.