rcheckin hanging when lots of renames in commit #190

Open
timabell opened this Issue Jul 19, 2012 · 11 comments

Comments

Projects
None yet
4 participants
Contributor

timabell commented Jul 19, 2012

I have no idea what is wrong, but I've been happily using the integration for a while now, and suddenly one of my commits just won't go to the tfs server. I've tried everything I can think of.

The command gets part way through the list of changes and then just hangs, I've waited ten mins or so and tried repeatedly. It doesn't always hang in the same place. It's a reasonably large commit with lots of additions, deletions and renames as well as normal diffs.

If you want any more diagnostics let me know. The output is:

$ git tfs rcheckin -i label -d
git command: Starting process: git log --no-color --pretty=medium HEAD
git command: Starting process: git config --list
git command time: [00:00:01.0515937] config --list
git command time: [00:00:01.2007931] log --no-color --pretty=medium HEAD
git command: Starting process: git diff-index --name-status -M HEAD
git command time: [00:00:00.4835907] diff-index --name-status -M HEAD
Fetching changes from TFS to minimize possibility of late conflict...
git command: Starting process: git log --no-color --pretty=medium refs/remotes/tfs/label
git command time: [00:00:00.1403973] log --no-color --pretty=medium refs/remotes/tfs/label
info: refs/remotes/tfs/label: Getting changesets from 1305 to current ...
git command: Starting process: git rev-list 77ed879acb73f60f0580ce396d4c307340239049 ^HEAD
git command time: [00:00:00.1247976] rev-list 77ed879acb73f60f0580ce396d4c307340239049 ^HEAD
git command: Starting process: git rev-list --parents --ancestry-path --first-parent --reverse 77ed879acb73f60f0580ce396d4c307340239049..HEAD
git command time: [00:00:00.1247976] rev-list --parents --ancestry-path --first-parent --reverse 77ed879acb73f60f0580ce396d4c307340239049..HEAD
git command: Starting process: git log 77ed879acb73f60f0580ce396d4c307340239049..4dde5f5895da8ddc3dbef4271a2eb79ee8266ae1
Starting checkin of 4dde5f58 'Reorganised local rebuild scripts.

-- snip --8<--

 delete database/utility-scripts/snapshot-LabelsQA.sql
Regex: ^:(?<srcmode>[0-7]{6}) (?<dstmode>[0-7]{6}) (?<srcsha1>[0-9a-f]{40}) (?<dstsha1>[0-9a-f]{40}) (?<status>.)(?<score>[0-9]*)\t(?<srcquot>"?)(?<srcpath>[^\t]+?)"?(\t(?<dstquot>"?)(?<dstpath>[^\t]+
?)"?)?$
Input: :100644 000000 23e3647bbbe849f5c4dde3e556643c15b9d84eaa 0000000000000000000000000000000000000000 D       database/utility-scripts/snapshot-ci.sql
 -> 0: >>:100644 000000 23e3647bbbe849f5c4dde3e556643c15b9d84eaa 0000000000000000000000000000000000000000 D     database/utility-scripts/snapshot-ci.sql<<
 -> 1: >><<
 -> srcmode: >>100644<<
 -> dstmode: >>000000<<
 -> srcsha1: >>23e3647bbbe849f5c4dde3e556643c15b9d84eaa<<
 -> dstsha1: >>0000000000000000000000000000000000000000<<
 -> status: >>D<<
 -> score: >><<
 -> srcquot: >><<
 -> srcpath: >>database/utility-scripts/snapshot-ci.sql<<
 -> dstquot: >><<
 -> dstpath: >><<
 delete database/utility-scripts/snapshot-ci.sql

and then it just hangs

Contributor

timabell commented Jul 20, 2012

Actually I think this might be nothing to do with git-tfs.

It did it to me again and I left it over night, when I came back it had exited with the following:

TF400324: Team Foundation services are not available from server http://xxxxx/xxx.
Technical information (for administrator):
  The operation has timed out
The operation has timed out

I wonder if this could be made easier to spot? I don't know how long it was before it timed out.

Contributor

timabell commented Jul 20, 2012

A couple of other data points, each rcheckin has been getting slower and slower over the couple of months and couple of hundred commits, and the two that failed have lots of renames in them.

Contributor

timabell commented Jul 20, 2012

Something else is wrong with this local repo it would seem as git tfs fetch is no longer getting the latest commits, this seems to have happened after the failure of the rcheckin.

I've tried cloning the git repo locally, then running git tfs bootstrap on the new repo, and then rcheckin for the troublesome commit, but the new copy hangs too.

Contributor

timabell commented Jul 20, 2012

I've tried again with a fresh repo generated with quick-clone, and that also hangs for a diff with 20+ file renames.

Contributor

timabell commented Jul 20, 2012

The workaround seems to be to do the renames manually using visual studio / tfs directly, check that in, then fetch with git-tfs.

Contributor

sc68cal commented Jul 24, 2012

Yes - large rename operations are checked back into TFS, by renaming/moving each file in the workspace. We need to investigate this and see if we can do the operations in parallel.

This is unusual though, that the process has hung. I've done very large renames, and the process would just print out each file being moved. I never encountered a hang.

Contributor

sc68cal commented Jul 24, 2012

If you have it happen again, would you be so kind as to start Visual Studio up, attach the VS debugger to the git-tfs process, pause it, and give us a stack trace? That way we could see where it is hanging.

Contributor

timabell commented Jul 24, 2012

Will do. Thanks for your comments.

Contributor

timabell commented Jul 25, 2012

Just for the record I just did quite a large commit with a fair number of renames in it with no problems at all (finished in 54 secs). My current suspicion is the stability of the TFS server.

Contributor

edwinf commented Jul 25, 2012

@timabell I've had issues with my own TFS server timing out on operations like that. I was helping Matt test out the workspace code he was writing by using one of my large instances and it always timed out during a clone. I did the same operation against TFSPreview and it worked fine. I then ended up proving it was my own server as I was able to cause the same issue to occasionally happen in team explorer. I think it's a problem with TFS running on a crappy win2003 server (at least in my case).

Contributor

jafin commented Jan 16, 2013

Git-tfs is freezing similar to what is described in the thread. It does seem to be related to large delete/rename checkins. Here's the stack trace if it helps. This is easily repeatable, the only workaround I have is to reduce the checkin to a smaller number of delete/renames, which sometimes succeeds.
I am talking to a TFS2008 server.

    [In a sleep, wait, or join] 
    >   mscorlib.dll!System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle waitableSafeHandle, long millisecondsTimeout, bool hasThreadAffinity, bool exitContext) + 0x22 bytes 
    mscorlib.dll!System.Threading.WaitHandle.WaitOne(int millisecondsTimeout, bool exitContext) + 0x28 bytes    
    System.dll!System.Net.LazyAsyncResult.WaitForCompletion(bool snap) + 0xd1 bytes 
    System.dll!System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest request, bool forcedsubmit) + 0x39d bytes  
    System.dll!System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest request, string connName) + 0x9b bytes   
    System.dll!System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint servicePoint) + 0x173 bytes  
    System.dll!System.Net.HttpWebRequest.GetRequestStream(out System.Net.TransportContext context) + 0x1e0 bytes    
    System.dll!System.Net.HttpWebRequest.GetRequestStream() + 0xe bytes 
    Microsoft.TeamFoundation.Client.dll!Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.SendRequestAndGetResponse(System.Net.HttpWebRequest webRequest, out System.Net.WebException webException) + 0x60 bytes   
    Microsoft.TeamFoundation.Client.dll!Microsoft.TeamFoundation.Client.Channels.TfsHttpWebRequest.SendRequest() + 0x61 bytes   
    Microsoft.TeamFoundation.Client.dll!Microsoft.TeamFoundation.Client.Channels.TfsHttpRequestChannel.Request(Microsoft.TeamFoundation.Client.Channels.TfsMessage message, System.TimeSpan timeout) + 0xe7 bytes   
    Microsoft.TeamFoundation.Client.dll!Microsoft.TeamFoundation.Client.Channels.TfsHttpClientBase.Invoke(Microsoft.TeamFoundation.Client.Channels.TfsClientOperation operation, object[] parameters, System.TimeSpan timeout, out object[] outputs) + 0x53 bytes   
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.Repository.PendChanges(string workspaceName, string ownerName, Microsoft.TeamFoundation.VersionControl.Client.ChangeRequest[] changes, int pendChangesOptions, int supportedFeatures, out Microsoft.TeamFoundation.VersionControl.Client.Failure[] failures) + 0xb8 bytes 
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.WebServiceLayer.PendChanges(string workspaceName, string ownerName, Microsoft.TeamFoundation.VersionControl.Client.ChangeRequest[] changes, int pendChangesOptions, int supportedFeatures, out Microsoft.TeamFoundation.VersionControl.Client.Failure[] failures, string[] itemPropertyFilters, string[] itemAttributeFilters, bool updateDisk, out bool onlineOperation, out int flags) + 0x1e4 bytes    
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.WebServiceLayerLocalWorkspaces.PendChanges(string workspaceName, string ownerName, Microsoft.TeamFoundation.VersionControl.Client.ChangeRequest[] changes, int pendChangesOptions, int supportedFeatures, out Microsoft.TeamFoundation.VersionControl.Client.Failure[] failures, string[] itemAttributeFilters, string[] itemPropertyFilters, bool updateDisk, out bool onlineOperation, out int flags) + 0x3da bytes 
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.Client.PendChanges(Microsoft.TeamFoundation.VersionControl.Client.Workspace workspace, Microsoft.TeamFoundation.VersionControl.Client.ChangeRequest[] changeRequests, bool updateDisk, Microsoft.TeamFoundation.VersionControl.Common.PendChangesOptions options, string[] itemAttributeFilters, string[] itemPropertyFilters, int operationId) + 0x9d bytes  
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.Workspace.PendChanges(Microsoft.TeamFoundation.VersionControl.Client.ChangeRequest[] changeRequests, bool updateDisk, Microsoft.TeamFoundation.VersionControl.Common.PendChangesOptions options, string[] itemAttributeFilters, string[] itemPropertyFilters) + 0x86 bytes    
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.Workspace.PendEdit(string[] paths, Microsoft.TeamFoundation.VersionControl.Client.RecursionType recursion, string fileType, Microsoft.TeamFoundation.VersionControl.Client.LockLevel lockLevel, bool silent, Microsoft.TeamFoundation.VersionControl.Common.PendChangesOptions options, string[] itemAttributeFilters, string[] itemPropertyFilters) + 0x81 bytes 
    Microsoft.TeamFoundation.VersionControl.Client.dll!Microsoft.TeamFoundation.VersionControl.Client.Workspace.PendEdit(string path, Microsoft.TeamFoundation.VersionControl.Client.RecursionType recursion) + 0x74 bytes  
    GitTfs.Vs11.dll!Sep.Git.Tfs.VsCommon.WrapperForWorkspace.PendEdit(string path) + 0x19 bytes 
    git-tfs.exe!Sep.Git.Tfs.Core.TfsWorkspace.Edit(string path) + 0x48 bytes    
    git-tfs.exe!Sep.Git.Tfs.Core.Changes.Git.RenameEdit.Apply(Sep.Git.Tfs.Core.ITfsWorkspace workspace) + 0x14 bytes    
    git-tfs.exe!Sep.Git.Tfs.Core.GitTfsRemote.PendChangesToWorkspace(string head, string parent, Sep.Git.Tfs.Core.ITfsWorkspace workspace) + 0x53 bytes 
    git-tfs.exe!Sep.Git.Tfs.Core.GitTfsRemote.CheckinTool(string head, Sep.Git.Tfs.Core.TfsChangesetInfo parentChangeset, Sep.Git.Tfs.Core.ITfsWorkspace workspace) + 0x45 bytes    
    git-tfs.exe!Sep.Git.Tfs.Core.GitTfsRemote.CheckinTool.AnonymousMethod__26(Sep.Git.Tfs.Core.ITfsWorkspace workspace) + 0x16 bytes    
    GitTfs.Vs11.dll!Sep.Git.Tfs.VsCommon.TfsHelperBase.WithWorkspace(string localDirectory, Sep.Git.Tfs.Core.IGitTfsRemote remote, Sep.Git.Tfs.Core.TfsChangesetInfo versionToFetch, System.Action<Sep.Git.Tfs.Core.ITfsWorkspace> action) + 0xe0 bytes 
    git-tfs.exe!Sep.Git.Tfs.Core.GitTfsRemote.CheckinTool(string head, Sep.Git.Tfs.Core.TfsChangesetInfo parentChangeset) + 0x81 bytes  
    git-tfs.exe!Sep.Git.Tfs.Commands.CheckinTool.DoCheckin(Sep.Git.Tfs.Core.TfsChangesetInfo changeset, string refToCheckin) + 0x2b bytes   
    git-tfs.exe!Sep.Git.Tfs.Commands.CheckinBase.PerformCheckin(Sep.Git.Tfs.Core.TfsChangesetInfo parentChangeset, string refToCheckin) + 0x1d bytes    
    git-tfs.exe!Sep.Git.Tfs.Commands.CheckinBase.Run.AnonymousMethod__0(Sep.Git.Tfs.Core.TfsChangesetInfo changeset) + 0xf bytes    
    git-tfs.exe!Sep.Git.Tfs.Core.TfsWriter.Write(string refToWrite, System.Func<Sep.Git.Tfs.Core.TfsChangesetInfo,int> write) + 0x8f bytes  
    git-tfs.exe!Sep.Git.Tfs.Commands.CheckinBase.Run(string refToCheckin) + 0x55 bytes  
    git-tfs.exe!Sep.Git.Tfs.Commands.CheckinBase.Run() + 0xc bytes  
    [Native to Managed Transition]  
    git-tfs.exe!Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(Sep.Git.Tfs.GitTfsCommand command, System.Collections.Generic.IList<string> args) + 0x189 bytes    
    git-tfs.exe!Sep.Git.Tfs.GitTfs.Main(Sep.Git.Tfs.GitTfsCommand command, System.Collections.Generic.IList<string> unparsedArgs) + 0x76 bytes  
    git-tfs.exe!Sep.Git.Tfs.GitTfs.Run(System.Collections.Generic.IList<string> args) + 0x56 bytes  
    git-tfs.exe!Sep.Git.Tfs.Program.Main(string[] args) + 0x5f bytes    

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment