-
Notifications
You must be signed in to change notification settings - Fork 4.9k
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of Repo.CommitsCount() #3022
Comments
This is even worse when viewing the "releases" page of a large repo with quite some releases as it gets called for every tag. Example repo with 162 releases(tags):
|
@DeX77 Nice catch, I missed that. All the more reason to cache the Repro is sitting up here if anyone wants to see this painful experience in action: https://git.uplinklabs.net/gogs/steven/linux |
Note that there is a rather new and unknown git feature called bitmap indexes (it think it is not activated by default) which already does exactly this: caching the number of commits reachable from a given commit and that could be used as a cache for rev-list --count. This article is an amazing introduction to this feature: http://githubengineering.com/counting-objects/ Note that once it is activated, rev-list --count automatically takes advantage of it. So fixing this issue could probably be as simple as activating bitmap indexes on the gogs side. |
Handy. I'm adding this to /etc/gitconfig on my git host and repacking all the repos to see if that helps:
|
Hmm, that didn't seem to help. I verified the bitmap was created:
But I don't think
|
I looked at the rev-list.c source and apparently there's an extra flag that makes it behave. Wonder if there's a config option too...
|
Great! So should this become the default configuration for Gogs? |
I don't think it's the full solution. The repository below is fully repacked and has the bitmap indexes. I've also patched gogs to add the https://git.uplinklabs.net/gogs/steven/linux This particular page loads significantly faster. Well... relatively. It's still very slow compared to e.g. GitHub. The "Releases" page is still completely inoperable though -- Chrome gives up waiting for it after a while. So it still seems like some caching mechanism is needed, even if that doesn't particularly help initial page load times. |
The commit count will make file history a problem too, e.g. via Repo.FileCommitsCount():
I think the simplest solution is to stop caring about the number of commits unless a way to cheaply calculate them can be found. In a lot of places, the rev-list count is only a mildly interesting statistic, without any practical use. |
Indeed, this info is mostly not very usefull anyway. Don't know if gogs can do that but what about doing that async and displaing it "when its done" ? |
I think that just shifts the pain from the end user to the server admin. The work still happens, it's just not apparent to the end user. Personally I'd rather not have my server spend the CPU cycles on it. |
Note: show Commits and Releases only in repositories home page as GitHub does. |
Merge this thread to #3518 |
@unknwon I don't see how the aesthetic design issue you referenced is relevant to a crippling performance issue. |
Hmm, you're right. |
The "Repo.CommitsCount" API really slows things down.
I have a mirror of Linus' linux.git tree, and viewing that in the gogs web interface is a miserable experience. It takes several seconds for it to render the page. I dug through and found that gogs was executing this command:
One way to improve this is to cache the constant data. You could permanently cache the
rev-list --count
values for tags, as that value would not change (as long as it's keyed by the tag hash rather than the tag name):So for the commit in question, the most recent tag in its ancestry is v4.6-rc5, and we could permanently cache the
rev-list --count
for the corresponding tag hash.Once the cache is populated, calculating the
rev-list --count
for an arbitary hash would go something like this:If you add that '3' to the tag's cached
rev-list --count
value, then you get the final CommitsCount(). You could cache that value as well, but give it a more sensible expiration.The text was updated successfully, but these errors were encountered: