Extremely slow UpdateAssemblyInfo in GitVersionTask #437
Comments
|
How big is your repository? i.e. what is the file size on disk? Are you getting a fresh clone each build? |
|
@gep13 The repository is 145 MB on disk and TeamCity is doing a clean checkout for each build to avoid random cache related bugs in MSBuild. |
|
Hmmm, so that isn't huge. Are the TeamCity server and Source Control Server on the same network? |
|
No, TeamCity is locally hosted while git is up on Bitbucket. However, GitVersionTask has run without problems in this and other repositories before, so I'm wondering if the branch layout combined with the last commit and something other internal crazy state causes GitVersion to get into a weird almost-eternal loop or something. The time consumption is happening between these two steps: |
|
Do you have the same issue locally? |
|
@JakeGinnivan Good question! I first tested the wrong branch, which of course didn't yield any slowness, but now that I've switched to the problematic branch, just loading the solution in Visual Studio takes ages. I don't see any output either, so I can't see exactly what is taking time. Could GitVersionTask affect the time it takes to load the project files it is included in? |
|
That sounds very strange. As a test, I would suggest removing GitVersionTask, i.e comment it out, and make sure there is not some other problem going on here. |
|
Went to get a ☕ and when I came back, VS was done loading the solution. The build is noticeably slower on the @gep13 Yea, I'll try that and report back the results. |
|
Just a sidenote, I discovered these strange things in the warning list in Visual Studio: |
|
Yes, removing GitVersionTask definitely sped up everything. Not just noticeably, but orders of magnitude. Reverting the changes with git so GitVersionTask was installed again, then doing "Reload All" in Visual Studio brought back the slowness in both project loading and the build. I've made a commit in the branch and pushed it so it triggers a TeamCity build. I'll report back on how long it takes for TeamCity to complete the build. |
|
@SimonCropp can you offer any suggestions on this one? |
|
can you share the repo? perhaps privately? Can you try an earlier version of GitVersion? |
|
I think I know what the issue is. For repos for large history I think the branch inheritance code is walking the git tree waaayy too much and is just thrashing the git repo.. On a repo/branch which causes this we need to debug and see what is happening in that function which is causing it to be so slow.. |
|
@JakeGinnivan That makes a lot of sense. This is an old project with probably several thousand commits. Couldn't the branch inheritance code break when it's (close to certain) about what to deduce from it to produce an accurate version number? Can you point to the piece of code, loop or otherwise in GitVersion where this is done? |
|
This will be the problematic method: |
|
It should look at all branches, find the oldest common commit and that be the hard stop |
|
Not much help, but we had a similar behavior with an early version of GV3 but then it was pretty obvious since it was +30000 commits in the meta data. Will test the repo again to see if it's possible to reproduce
|
|
@JakeGinnivan So the code that might be causing the slowness can be reduced to these few lines: excludedBranches.ToList().ForEach(excludedInheritBranches.Add);
var branchPoint = currentBranch.FindCommitBranchWasBranchedFrom(repository, onlyEvaluateTrackedBranches, excludedInheritBranches.ToArray());
List<Branch> possibleParents;
if (branchPoint.Sha == currentCommit.Sha)
{
possibleParents = currentCommit.GetBranchesContainingCommit(repository, true).Except(excludedInheritBranches).ToList();
}
else
{
var branches = branchPoint.GetBranchesContainingCommit(repository, true).Except(excludedInheritBranches).ToList();
var currentTipBranches = currentCommit.GetBranchesContainingCommit(repository, true).Except(excludedInheritBranches).ToList();
possibleParents = branches
.Except(currentTipBranches)
.ToList();
} |
|
@asbjornu have you tested with these changes? Did it fix your problem? |
|
@SimonCropp, @gep13, Downgrading to version @JakeGinnivan, One thing GitVersion should do regardless of the performance of |
|
@asbjornu I know there are caching things it does locally already, so that might happen. @SimonCropp may know more At any rate, it should be super fast to do this. I didn't even think about perf when writing it unfortunately |
|
@JakeGinnivan The build output I pasted originally is repeated for every project in the solution with the same hop in time between |
|
Since adding MiniProfiler to GitVersionTask have brought me into a However, when I ran it in the same solution, Here's my profiling branch if @JakeGinnivan, @gep13, @SimonCropp or anyone else wants to have a look at it. For completeness, here's the profile results from GitVersion.exe: If you could just help me get MiniProfiler working in GitVersionTask, I can get profile results for it too. The problem is that MSBuild complains that it can't find MiniProfiler when I execute the GitVersionTask that has the MiniProfiler reference: |
|
This should be fixed now, I was doing something very stupid when trying to find the source branches. It now should be fast. Reopen/create a new issue if it's still an issue (test against master) |
|
@JakeGinnivan I've tried to make a custom build and replace the local GitVersionTask package, and it looks better, but still slower than on version 2. To be sure, it would be great with a beta 3 build on NuGet. Would that be possible to get out the door anytime soon? |
|
We need to add logging then so we can see what is taking the time. I can still see stuff which would be slow though |
|
@JakeGinnivan I've started a branch that adds MiniProfiler, but it got kind of borked after a rebase, so I can redo that if you'd like? If profiling should be merged into master, some kind of abstraction should be added on top of MiniProfiler, so GitVersion doesn't expose it as a requirement. |
|
If we add more logging, and the logging logs out timestamps we should have a pretty good idea what is slow without adding profiling in. Up to you though |
|
@JakeGinnivan True. It's just that MiniProfiler keeps track of nesting and provides nice formatting with indentation and such, as you can see here. I'll see what I can do. |
|
@asbjornu I have pushed v3-beta.3. Let me know how you go |
|
@JakeGinnivan Thanks, I've updated and tried again. While performance is much better, version 3 is still quite a bit slower than version 2. A clear problem still is that the version number isn't successfully cached between projects within the same solution. This shows up in the build log for all projects in the solution on TeamCity: This of course leads to GitVersion doing fetch and various other things it shouldn't need to do more than once for the entire solution. The |
|
Ok cool, still work to do but getting there it seems. 24 seconds is still no where near acceptable, it should be <1 second to run. Any thoughts on a big oss repo which shows off the issues so I can test? |
|
@JakeGinnivan I expect that GitVersionTask's performance in NServiceBus is impeccable? If not, that's a pretty obvious place to start. :) Then you have NHibernate, of course. No idea if it actually shows any of the problems I have, but let's hope it does. :) |
|
@asbjornu I have just merged logging improvements into the release/3.0.0 branch It adds basic timing information and scoping, I am trying to close a few of the issues over this weekend so I will try to profile on a larger repo but just letting you know if you want to look into it as well :) |
|
@JakeGinnivan Awesome! How do I enabled more verbose logging output when building in Visual Studio? |
|
@JakeGinnivan, I now built with MSBuild and got the following output: While things look much better, |
|
Will have to look into it some time, not that familiar with the MSBuild task. Need to change that |
|
@JakeGinnivan I think it would speed things up considerably if the version number was actually cached between projects in a solution. Should I open a separate bug for that? |
|
Yeah, it will allow someone to pick it up and sort that issue in isolation |
|
The By contrast, the git command line just takes less than 1 second to get the branches that contains a commit. I'm new to gitversion and libgit2sharp's code. Wondering if it's possible to use git command line rather than libgit2sharp, or maybe there's some equivalent APIs in libgit2sharp we can call? |
|
@EbenZhang If Without such a venture, #711 should at least ensure that |
|
libgit2sharp has the below code to get the LOCAL branches for a commit: I'm not sure how is the performance, and not sure how to get the REMOTE branches. I've implemented using git command to get the branches for a commit and tag, it helps to reduce the time from 20min(2hours worst case) to 1 minute. I'll clean up my code, do more tests, probably need to fix the unit tests as well, and then create a pull request (hopefully this week) |
|
@EbenZhang Excellent! Looking forward to seeing the PR! 😄 👍 |
|
See #734 |
I'm using GitVersionTask 3.0.0-beta0002 in a project that suddenly experience extreme slowness in the
UpdateAssemblyInfotask. For each project having a GitVersionTask reference, this task takes well over 2 minutes. Here's some relevant output from the TeamCity build log:As far as I can tell, there's no external dependencies (like network resources) that should take any time here as most of the time is spent to complete the following step:
If this step is dependent on network resources and my network for some reason is unbearably slow at the moment for this exact operation but not anything else, I understand that this might be impossible to debug or fix, but I would love any insight into why my build suddenly went from taking ~5 minutes to now completing in 1 hour and 20 minutes.
The text was updated successfully, but these errors were encountered: