Subtree command for git-tfs #350

Merged
merged 19 commits into from Sep 12, 2013

Projects

None yet

5 participants

@gburgett

I've written an implementation for issue #344.

When executing the following command:

git-tfs subtree add -p=[prefix] [tfs-url] [repo-path]

git-tfs will:

  • Locate or create an "owner" remote for the subtree, this remote has no TfsRepositoryPath but rather delegates to other remotes which represent its subtrees. Uses "default" unless instructed otherwise.
  • Create a subtree remote with ID `[owner]_subtree/[prefix]
  • Set the subtree remote's workspace to be inside the owning remote's workspace, at workspace/[prefix]
  • Fetch the subtree remote
  • execute git subtree add --prefix=[prefix] [subtree remote] -m [commit msg], where the commit message includes a git-tfs-id line for the subtree master.

Afterwards, the revision history will look like this:

commit e75165c22d0415613129cbc5456cc7b491ec6903
Merge: 845abef 5618bb9
Author: Gordon Burgett <my.email@gmail.com>
Date:   Wed Apr 10 13:38:32 2013 -0500

     Add 'InternalTools/' from commit '5618bb9065d9df8b059e7218db1a639e38a54f22'

    git-tfs-id: [http://my.server.url:8080/tfs/myco];C19373

    git-subtree-dir: InternalTools
    git-subtree-mainline: 845abef174122eb5f5985d899a3875be965654c7
    git-subtree-split: 5618bb9065d9df8b059e7218db1a639e38a54f22

commit 5618bb9065d9df8b059e7218db1a639e38a54f22
Author: Someone Else <someone.else@company.com>
Date:   Mon Apr 8 21:20:27 2013 +0000

    Fixes #1094 - some other task

    git-tfs-id: [http://my.server.url:8080/tfs/myco]$/Production/InternalTools/MAIN;C19370

From this point on, a git-tfs pull against the owning remote ("default") will pull all changesets across all known subtree remotes (in this case "default_subtree/InternalTools") and will apply the changesets.
A git-tfs checkin will pend all changes across all subtree remotes in the same workspace.
Similarly with a shelve or unshelve.

Things left to do:

  • Modify Init-Branch to handle subtrees
  • Add handling for pushing and pulling to a single subtree remote from within a subtree branch, for example after executing git subtree split --prefix=InternalTools I should be able to pull from "default_subtree/InternalTools" and checkin to the same.
gburgett added some commits Apr 4, 2013
@gburgett gburgett added subtree command for "subtree add" and "subtree pull" c8300df
@gburgett gburgett TfsHelper can now map multiple paths in a workspace, remotes now map …
…all paths for all subtrees

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
fc1365d
@gburgett gburgett Fixing internals for when a remote has subtrees
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
4cfe68b
@gburgett gburgett subtree now updates the owning remote to the commit after the subtree…
… merge
d67e985
@gburgett gburgett in-progress getting checkin to work, need to figure out why it wont m…
…ap the workspace
fbb1d8d
@gburgett gburgett checkin works for remote with subtrees
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
a3733a8
@gburgett gburgett added "subtree split", which splits and also advances TFS refs as nec…
…essary

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
664202e
@gburgett gburgett fixed issue with pull - GetPathInGitRepo was not taking prefix into a…
…ccount

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
2eff0e2
@gburgett gburgett Master remotes now delegate to subtrees when getting local item for s…
…erver item. This fixed an issue with unshelve.

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
8af8a41
@sc68cal sc68cal and 1 other commented on an outdated diff Apr 14, 2013
GitTfs/Commands/Subtree.cs
+ }
+
+ return GitTfsExitCodes.OK;
+ }
+
+ private void ValidatePrefix()
+ {
+ if (!Directory.Exists(Prefix))
+ {
+ throw new GitTfsException(string.Format("Directory {0} does not exist", Prefix))
+ .WithRecommendation("Add the subtree using 'git tfs subtree add -p=<prefix> [tfs-server] [tfs-repository]'");
+ }
+ }
+
+
+ private void command(List<string> args)
@sc68cal
sc68cal Apr 14, 2013

👎 We do not need yet another wrapper around shell commands.

@gburgett
gburgett Apr 14, 2013

Certainly, I was using this for debugging, I'll get rid of it.

@sc68cal sc68cal and 1 other commented on an outdated diff Apr 14, 2013
GitTfs/Core/TfsWriter.cs
{
case 1:
- var changeset = tfsParents.First();
+ var changeset = tfsParents[0];
@sc68cal
sc68cal Apr 14, 2013

I think this was fine the way it was.

@gburgett
gburgett Apr 14, 2013

I'm willing to revert it to using LINQ & enumerable only if you prefer. I need this to be recursive because we could have the situation where there is a mixture of changesets from a "subtree owner" remote as well as multiple subtrees, If we encounter that situation we ought to just take the "subtree owner" remote.

gburgett added some commits Apr 15, 2013
@gburgett gburgett Distinct operation on FetchChangesets was causing parts of changesets…
… to be missed when a changeset spanned multiple remote projects.

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
1c54f69
@gburgett gburgett making changes proposed in pull request
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
74fe7dd
@sc68cal

👍 for addressing my nitpicks 😄

gburgett added some commits Apr 16, 2013
@gburgett gburgett Owning remote now uses its own history for fetching new changesets ra…
…ther than relying on the subtree branches.

The owning remote had been pulling changesets multiple times because FetchChangesets() was using the wrong MaxChangesetId

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
5de2db1
@gburgett gburgett GetPathInGitRepo was not working correctly when going directly throug…
…h a subtree remote.

The prefix only needs to be applied when working from the context of the owner remote.

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
e8ddb52
@gburgett gburgett Used the wrong remote for the prefix in GetPathInGitRepo
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
ccbb0f7
@spraints
git-tfs member

I'm still thinking about this. I like the general idea of this. My main hangup is on how it's configured. The subtree command is ok, but I don't like polluting commit messages with more git-tfs-x stuff. One of my rules of thumb about git-tfs is "anyone who clones a TFS repo with the same command should end up with the exact same git repo." That starts to break down if you use things like git tfs checkin or an author map. But the more that we can keep all of the "what makes this repo have its special ❄️ shape" stuff in the git tfs clone command, the happier I'll be. Maybe we merge the subtree things into options for clone?

Also, there's the problem of synchronizing branches across subtrees. Are branches always in lock-step across the sub-projects? If there's a branch in one but not the other, how do we know which to use?

Just for the mental exercise, I'm going to sketch some command line interface examples:

With git-tfs master, you'd have to create separate repositories, and maybe link them via a submodule:

> git tfs clone http://myserver $/Production/Product1/MAIN product1
> git tfs clone http://myserver $/Production/InternalTools/MAIN internal-tools
> cd product1
> git submodule add ../internal-tools InternalTools

With the code on this branch, you'd do this:

> git tfs clone http://myserver $/Production/Product1/MAIN product1
> cd product1
> git tfs subtree add -p=InternalTools http://myserver $/Production/InternalTools/MAIN InternalTools

With an all-in-one clone:

> git tfs clone http://tfs/blah new-git-repo --map $/Production/Product1/MAIN= --map $/Production/InternalTools/MAIN=InternalTools

I wonder if subtrees and branches should be mutually exclusive? It seems like the complexity of managing both in git-tfs at the same time is not going to be fun. So, if you have branches, I would recommend that you use multiple repositories and git submodules. If you just have a weird repository setup (like, if you just care about the MAIN branches of each subdir in the #344 example (I'm not saying you're weird, or that your repository is weird, but .. just that ... um... anyways)) then we offer some kind of mapping?

I dunno. Like I said, I'm still trying to work through this. By all means, start using this, and let us know if you get into it and it works out really rock solid awesome, or whatever. Real experience is a trump card.

@gburgett

Thank you spraints,

I understand where you're going with the idea "anyone who clones a TFS repo with the same command should end up with the exact same git repo." I like that as well. Ideally, having it as a subtree command would adhere to that. Git's subtree command is designed to create the same "artificial" revision history for a subtree each time the subtree is split, but it is potentially destructive to that if you do a remerge (see documentation here).

I do agree with you that the complexity of managing both subtrees and branches in the same repo is likely prohibitive, but I want to try, since having multiple branches in the same workspace is one of the main reasons I would like to use git-tfs. I think it is possible if we improve the subtree command, and I will continue to try as I have time.

One of the advantages of continuing to mimic git's subtrees would be to have the ability to do a subtree split and then merge or checkin an individual subtree remote from the split git branch. This could be a very powerful tool, if also very complex. Another advantage would be the ability to include a TFS project in an existing non-TFS git repo as a subtree, rather than a submodule.

I am continuing to use this branch, and will continue to improve it as I find bugs. I have not had a chance to attempt some of the more complex options with this, such as doing an init-branch with subtrees (frankly I am a little scared to do so :) ), but I'll definitely be trying once I trim down my pending changes.

gburgett and others added some commits Apr 24, 2013
@gburgett gburgett The local paths for server paths ought to be controlled by the worksp…
…ace mapping, not the remote doing the command. Before subtree, these were always the same, but now that is no longer the case.

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
eb4115b
Stephen Baynham Renames now get the correct workspace path when pulling the renamed file
from TFS.
fb0f19a
@gburgett gburgett Subtree remote's git-tfs ID is now applied when pulling new changeset…
…s, TfsWriter now handles determining whether to write with subtree or its owner.

This facilitates using git-tfs after having performed a subtree-split.  In the subtree's artificial revision history, the owner's git-tfs ID will not be present, so the TfsWriter will proceed as though the subtree remote were a normal remote.

Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
c696499
@gburgett

I'm still working on this. One of the things I just added was support for when you do a native "git subtree split". Git-tfs will treat the split revision history as it would for a normal repository, since the split revision history should only have one TFS parent (the subtree remote). This allows doing a subtree split then a git-tfs pull/checkin.

It seems like that would make it fairly easy to get out of sync with other people's local repositories, but I think that is a factor of the git subtree command and not git-tfs' handling of it. In doing this my philosophy has been to try to mirror as closely as possible git's native subtree command.

@sc68cal

@gburgett Just wanted to bring this back up - you've stated that you're using this branch in your workflow. Would you like to rebase / fix conflicts and get it merged? If you're using it and it scratches your itch - I'm OK with merging.

@gburgett

Sure thing. I've been using it for a while now, it works great for fetch/pull, shelve, and checkin. I haven't tested it with branching, I haven't had time.

@sc68cal

OK - can you just updated it with the latest from master, fix any conflicts, and then we'll merge it?

gburgett added some commits Sep 5, 2013
@gburgett gburgett Merge branch 'master' into subtree
Conflicts:
	GitTfs.VsCommon/TfsHelper.Common.cs
	GitTfs/Core/GitRepository.cs
	GitTfs/Core/GitTfsRemote.cs
	GitTfs/Core/IGitRepository.cs
	GitTfs/Core/TfsInterop/ITfsHelper.cs
85d85d4
@gburgett gburgett Fixed other conflicts, all tests passing
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
049be97
@gburgett

Alright I've merged in from master. Subtree add, shelve, checkintool, and pull are all working nicely, and it plays well with regular git subtree as well.

@spraints spraints merged commit 5398544 into git-tfs:master Sep 12, 2013

1 check passed

Details default The Travis CI build passed
@spraints
git-tfs member

💥 thanks!

@patthoyts

This merge has broken the tree. Using VS 2010 I get a number of errors building with msbuild GitTfs.sln /t:Rebuild /p:Configuration=Release (also with Debug). The problem seems to be a missing assembly reference now required by TfsHelper.Common.cs.

@pmiossec
git-tfs member

@patthoyts
The problem seems to be a missing assembly reference now required by TfsHelper.Common.cs
No. There is just 2 errors in the code.

Should correct the build :
https://github.com/pmiossec/git-tfs/tree/subtree_correct_build

but I don't guaranty that my correction is the good one (but should be...).

If @gburgett could have a look at that... and do a pull request quickly!

@spraints spraints added a commit that referenced this pull request Sep 12, 2013
@spraints spraints Revert "Merge pull request #350 from gburgett/subtree"
This reverts commit 5398544, reversing
changes made to 853c733.
52ea152
@spraints spraints added a commit that referenced this pull request Sep 12, 2013
@spraints spraints Resolve build errors introduced by #350 c040dc7
@spraints
git-tfs member

@patthoyts Thanks for the heads up! @pmiossec Thanks for the quick fix! (It was better than my knee jerk revert :P).

@pmiossec
git-tfs member

not sure my fix is the good one.
Especially the line :
workspace = GetWorkspace(new WorkingFolder(projectPath, gitRepositoryPath));
I hope @gburgett could give us an answer quickly...

@spraints
git-tfs member

@pmiossec I merged it to get master to build cleanly. Your fix looks good to me, but I would also like @gburgett's feedback.

@gburgett

I'll take a look and make sure it still works with my use case

thanks!

@gburgett gburgett added a commit to gburgett/git-tfs that referenced this pull request Sep 12, 2013
@gburgett gburgett #350 - fixes nullref due to build errors in the merged pull request
Signed-off-by: Gordon Burgett <gordon.burgett@gmail.com>
58a1111
@gburgett

When I opened up my local repo, I noticed the one file I had forgotten to add to the checkin doh

I've got another commit to address a nullref in @pmiossec's fix, I've got it on my subtree branch here:
gburgett@58a1111

@pmiossec
git-tfs member
@gburgett

good plan, i've opened #435

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment