Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] merge --autostash: apply autostash in more cases #1002

Closed

Conversation

phil-blain
Copy link

This series aims to apply the stash created by 'git merge --autostash' in 2
situations that were not covered by the code:

  1. If the merge is fast-forward but
    the fast-forward operation fails - PATCH 3/4
  2. If the merge strategy completely fails
    to handle the merge (exit code 2) - PATCH 4/4

The first 2 commits are small improvements that I noticed while implementing
the other two.

I'm marking it [RFC] because I'm not 100% sure that trying to apply the
autostash in 3/4 and 4/4 is actually the best course of action (or if
it would be better to call 'save_autostash' instead). That's because:

For 3/4 (fast-forward fails):
I'm not sure if 'unpack_trees' (called by 'checkout_fast_forward') is
guaranteed to fail atomically, or it might fail mid-way and leave the worktree
unclean, in which case it might be better not to apply the autostash, but
just save it instead (and tell the user). In the test case I'm adding, it does
fail before starting to update the working tree, but I'm not sure if it's always
the case.

For 4/4 (merge strategy fails):
Same reasoning: I'm not sure if all strategies (and even less user-supplied ones,
which are not documented but kind of supported) are guaranteed to 'exit 2' before
messing up the working tree.

CC: Denton Liu liu.denton@gmail.com
CC: Felipe Contreras felipe.contreras@gmail.com

The variable 'best_strategy' holds the name of the merge strategy that
resulted in fewer conflicts, if several strategies were tried. When
that's the case but the best strategy was not the first one tried, we
inform the user which strategy was the "best" one before recreating the
merge and leaving the conflicted files in the tree.

This informational message is missing the word "strategy", so it shows
something like:

    Using the recursive to prepare resolving by hand.

Fix that.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
The documentation for 'git merge --abort' and 'git merge --quit' both
mention the special ref 'MERGE_AUTOSTASH', but this ref is not formally
defined anywhere. Mention it in the description of the '--autostash'
option for 'git merge'.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Since 'git merge' learned '--autostash' in a03b555 (merge: teach
--autostash option, 2020-04-07), 'cmd_merge', in the fast-forward case,
calls 'create_autostash' before calling 'checkout_fast_forward' if
'--autostash' is given.

However, if 'checkout_fast_forward' fails, the autostash is not applied
to the working tree, nor saved in the stash list, since the code simply
calls 'goto done'.

Be more helpful to the user by applying the autostash in that case.

An easy way to test a failing fast-forward is when we are merging a
branch that has a tracked file that conflicts with an untracked file in
the working tree.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
Since 'git merge' learned '--autostash' in a03b555 (merge: teach
--autostash option, 2020-04-07), 'cmd_merge', once it is determined that
we have to create a merge commit, calls 'create_autostash' if
'--autostash' is given.

As explained in a03b555, and made more abvious by the tests added in
that commit, the autostash is then applied if the merge succeeds, either
directly or by committing (after conflict resolution or if '--no-commit'
was given), or if the merge is aborted with 'git merge --abort'. In some
other cases, like the user calling 'git reset --merge' or 'git merge
--quit', the autostash is not applied, but saved in the stash list.

However, there exists a scenario that creates an autostash but does not
apply nor save it to the stash list: if the chosen merge strategy
completely fails to handle the merge, i.e. 'try_merge_strategy' returns
2.

Apply the autostash in that case also. An easy way to test that is to
try to merge more than two commits but explicitely ask for the 'recursive'
merge strategy.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
@phil-blain
Copy link
Author

/submit

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 23, 2021

Submitted as pull.1002.git.1627042470.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git pr-1002/phil-blain/merge-autostash-ff-fails-v1

To fetch this version to local tag pr-1002/phil-blain/merge-autostash-ff-fails-v1:

git fetch --no-tags https://github.com/gitgitgadget/git tag pr-1002/phil-blain/merge-autostash-ff-fails-v1

@@ -1715,7 +1715,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
else {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Git mailing list, Felipe Contreras wrote (reply to this):

Philippe Blain via GitGitGadget wrote:
> From: Philippe Blain <levraiphilippeblain@gmail.com>
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -1715,7 +1715,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
>  	else {
>  		printf(_("Rewinding the tree to pristine...\n"));
>  		restore_state(&head_commit->object.oid, &stash);
> -		printf(_("Using the %s to prepare resolving by hand.\n"),
> +		printf(_("Using the %s strategy to prepare resolving by hand.\n"),

Obviously correct.

-- 
Felipe Contreras

@@ -154,7 +154,8 @@ endif::git-pull[]
--autostash::
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Git mailing list, Felipe Contreras wrote (reply to this):

Philippe Blain via GitGitGadget wrote:
> From: Philippe Blain <levraiphilippeblain@gmail.com>

> --- a/Documentation/merge-options.txt
> +++ b/Documentation/merge-options.txt
> @@ -154,7 +154,8 @@ endif::git-pull[]
>  --autostash::
>  --no-autostash::
>  	Automatically create a temporary stash entry before the operation
> -	begins, and apply it after the operation ends.  This means
> +	begins, record it in the special ref `MERGE_AUTOSTASH`
> +	and apply it after the operation ends.  This means
>  	that you can run the operation on a dirty worktree.  However, use
>  	with care: the final stash application after a successful
>  	merge might result in non-trivial conflicts.

Although normally I'm against mentioning low-level technical details in
high-level documentation, there doesn't seem to be a better location to
explain MERGE_AUTOSTASH, and it's just a simple sentence.

Fine by me.

-- 
Felipe Contreras

@@ -1560,6 +1560,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
&head_commit->object.oid,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Git mailing list, Felipe Contreras wrote (reply to this):

Philippe Blain via GitGitGadget wrote:
> From: Philippe Blain <levraiphilippeblain@gmail.com>

> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -1560,6 +1560,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
>  					  &head_commit->object.oid,
>  					  &commit->object.oid,
>  					  overwrite_ignore)) {
> +			apply_autostash(git_path_merge_autostash(the_repository));
>  			ret = 1;
>  			goto done;
>  		}

I can verify that this fixes my simple test.

> diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> index 1cbc9715a81..216113d3537 100755
> --- a/t/t7600-merge.sh
> +++ b/t/t7600-merge.sh
> @@ -122,6 +122,8 @@ test_expect_success 'setup' '
>  	c0=$(git rev-parse HEAD) &&
>  	cp file.1 file &&
>  	git add file &&
> +	cp file.1 other &&
> +	git add other &&
>  	test_tick &&
>  	git commit -m "commit 1" &&
>  	git tag c1 &&
> @@ -711,6 +713,15 @@ test_expect_success 'fast-forward merge with --autostash' '
>  	test_cmp result.1-5 file
>  '
>  
> +test_expect_success 'failed fast-forward merge with --autostash' '
> +	git reset --hard c0 &&
> +	git merge-file file file.orig file.5 &&
> +	cp file.5 other &&
> +	test_must_fail git merge --autostash c1 2>err &&
> +	test_i18ngrep "Applied autostash." err &&

I've heard others test we are moving away from test_i18ngrep in favor of
grep.

> +	test_cmp file.5 file
> +'
> +
>  test_expect_success 'octopus merge with --autostash' '
>  	git reset --hard c1 &&
>  	git merge-file file file.orig file.3 &&

-- 
Felipe Contreras

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 23, 2021

On the Git mailing list, Felipe Contreras wrote (reply to this):

Philippe Blain via GitGitGadget wrote:
> This series aims to apply the stash created by 'git merge --autostash' in 2
> situations that were not covered by the code:
> 
>  1. If the merge is fast-forward but the fast-forward operation fails -
>     PATCH 3/4
>  2. If the merge strategy completely fails to handle the merge (exit code 2)
>     - PATCH 4/4
> 
> The first 2 commits are small improvements that I noticed while implementing
> the other two.
> 
> I'm marking it [RFC] because I'm not 100% sure that trying to apply the
> autostash in 3/4 and 4/4 is actually the best course of action (or if it
> would be better to call 'save_autostash' instead). That's because:
> 
> For 3/4 (fast-forward fails): I'm not sure if 'unpack_trees' (called by
> 'checkout_fast_forward') is guaranteed to fail atomically, or it might fail
> mid-way and leave the worktree unclean, in which case it might be better not
> to apply the autostash, but just save it instead (and tell the user). In the
> test case I'm adding, it does fail before starting to update the working
> tree, but I'm not sure if it's always the case.

I'm not familiar with unpack_trees, but sans that possibility the whole
series looks fine to me.

-- 
Felipe Contreras

@@ -1560,6 +1560,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
&head_commit->object.oid,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Philippe Blain <levraiphilippeblain@gmail.com>
>
> Since 'git merge' learned '--autostash' in a03b55530a (merge: teach
> --autostash option, 2020-04-07), 'cmd_merge', in the fast-forward case,
> calls 'create_autostash' before calling 'checkout_fast_forward' if
> '--autostash' is given.
>
> However, if 'checkout_fast_forward' fails, the autostash is not applied
> to the working tree, nor saved in the stash list, since the code simply
> calls 'goto done'.
>
> Be more helpful to the user by applying the autostash in that case.
>
> An easy way to test a failing fast-forward is when we are merging a
> branch that has a tracked file that conflicts with an untracked file in
> the working tree.

I think this one makes sense, as the case that is tested I know
fast-forward (aka two-tree switching) would be atomic. If your
working tree is already broken (e.g. triggers I/O errors in the
middle, some directories having wrong permissions to make them
unwritable by you, etc.), you would also see an error from
fast-forward and you probably cannot tell these two cases apart,
and trying to unstash the local changes may make things even worse,
but I suspect that there isn't much you can do about it.

Thanks.

>
> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>  builtin/merge.c  |  1 +
>  t/t7600-merge.sh | 11 +++++++++++
>  2 files changed, 12 insertions(+)
>
> diff --git a/builtin/merge.c b/builtin/merge.c
> index 74797b6c7a6..788a6b0cd55 100644
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -1560,6 +1560,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
>  					  &head_commit->object.oid,
>  					  &commit->object.oid,
>  					  overwrite_ignore)) {
> +			apply_autostash(git_path_merge_autostash(the_repository));
>  			ret = 1;
>  			goto done;
>  		}
> diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> index 1cbc9715a81..216113d3537 100755
> --- a/t/t7600-merge.sh
> +++ b/t/t7600-merge.sh
> @@ -122,6 +122,8 @@ test_expect_success 'setup' '
>  	c0=$(git rev-parse HEAD) &&
>  	cp file.1 file &&
>  	git add file &&
> +	cp file.1 other &&
> +	git add other &&
>  	test_tick &&
>  	git commit -m "commit 1" &&
>  	git tag c1 &&
> @@ -711,6 +713,15 @@ test_expect_success 'fast-forward merge with --autostash' '
>  	test_cmp result.1-5 file
>  '
>  
> +test_expect_success 'failed fast-forward merge with --autostash' '
> +	git reset --hard c0 &&
> +	git merge-file file file.orig file.5 &&
> +	cp file.5 other &&
> +	test_must_fail git merge --autostash c1 2>err &&
> +	test_i18ngrep "Applied autostash." err &&
> +	test_cmp file.5 file
> +'
> +
>  test_expect_success 'octopus merge with --autostash' '
>  	git reset --hard c1 &&
>  	git merge-file file file.orig file.3 &&

@@ -1560,6 +1560,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
&head_commit->object.oid,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Philippe Blain via GitGitGadget" <gitgitgadget@gmail.com> writes:

> As explained in a03b55530a, and made more abvious by the tests added in
> that commit, the autostash is then applied if the merge succeeds, either
> directly or by committing (after conflict resolution or if '--no-commit'
> was given), or if the merge is aborted with 'git merge --abort'. In some
> other cases, like the user calling 'git reset --merge' or 'git merge
> --quit', the autostash is not applied, but saved in the stash list.

OK.

> However, there exists a scenario that creates an autostash but does not
> apply nor save it to the stash list: if the chosen merge strategy
> completely fails to handle the merge, i.e. 'try_merge_strategy' returns
> 2.
>
> Apply the autostash in that case also. An easy way to test that is to
> try to merge more than two commits but explicitely ask for the 'recursive'
> merge strategy.

We know that the recursive that barfs when asked to merge more than
one "other" histories will fail before touching the index or the
working tree, but I do not think the merge-strategy protocol makes
any guarantee as to what half-modified state the strategy would
leave the index and the working tree in.

So this change at the first glance looks risky, but at least we have
a call to restore_state() before you added an apply_autostash().
The restore_state() call is the same as the one is used between
application of a strategy that has failed the same way in the loop,
so we should be able to trust it to bring the state back to the
pristine (i.e. after autostash removed the local changes).

I think this change makes sense, too.

(I didn't look at 1/4 and 2/4)

Thanks.



> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>  builtin/merge.c  | 1 +
>  t/t7600-merge.sh | 8 ++++++++
>  2 files changed, 9 insertions(+)
>
> diff --git a/builtin/merge.c b/builtin/merge.c
> index 788a6b0cd55..d44c14a21a3 100644
> --- a/builtin/merge.c
> +++ b/builtin/merge.c
> @@ -1709,6 +1709,7 @@ int cmd_merge(int argc, const char **argv, const char *prefix)
>  		else
>  			fprintf(stderr, _("Merge with strategy %s failed.\n"),
>  				use_strategies[0]->name);
> +		apply_autostash(git_path_merge_autostash(the_repository));
>  		ret = 2;
>  		goto done;
>  	} else if (best_strategy == wt_strategy)
> diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
> index 216113d3537..2ef39d3088e 100755
> --- a/t/t7600-merge.sh
> +++ b/t/t7600-merge.sh
> @@ -732,6 +732,14 @@ test_expect_success 'octopus merge with --autostash' '
>  	test_cmp result.1-3-5-9 file
>  '
>  
> +test_expect_success 'failed merge (exit 2) with --autostash' '
> +	git reset --hard c1 &&
> +	git merge-file file file.orig file.5 &&
> +	test_must_fail git merge -s recursive --autostash c2 c3 2>err &&
> +	test_i18ngrep "Applied autostash." err &&
> +	test_cmp result.1-5 file
> +'
> +
>  test_expect_success 'conflicted merge with --autostash, --abort restores stash' '
>  	git reset --hard c3 &&
>  	cp file.1 file &&

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 23, 2021

This branch is now known as pb/merge-autostash-more.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 23, 2021

This patch series was integrated into seen via git@c49ebfe.

@gitgitgadget gitgitgadget bot added the seen label Jul 23, 2021
@gitgitgadget
Copy link

gitgitgadget bot commented Jul 26, 2021

This patch series was integrated into seen via git@d1001de.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 26, 2021

This patch series was integrated into seen via git@e828d8a.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 27, 2021

This patch series was integrated into seen via git@1d9f86b.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 27, 2021

This patch series was integrated into seen via git@8dd7d1f.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 27, 2021

There was a status update in the "New Topics" section about the branch pb/merge-autostash-more on the Git mailing list:

The local changes stashed by "git merge --autostash" were lost when
the merge failed in certain ways, which has been corrected.

Will merge to 'next'.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 27, 2021

This patch series was integrated into seen via git@3216ce1.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 28, 2021

This patch series was integrated into seen via git@e496b42.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 28, 2021

This patch series was integrated into seen via git@8b56c55.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 28, 2021

This patch series was integrated into seen via git@fa0a580.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 29, 2021

There was a status update in the "Cooking" section about the branch pb/merge-autostash-more on the Git mailing list:

The local changes stashed by "git merge --autostash" were lost when
the merge failed in certain ways, which has been corrected.

Will merge to 'next'.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 30, 2021

This patch series was integrated into seen via git@e7de694.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 30, 2021

This patch series was integrated into next via git@bfc8b41.

@gitgitgadget gitgitgadget bot added the next label Jul 30, 2021
@gitgitgadget
Copy link

gitgitgadget bot commented Aug 2, 2021

There was a status update in the "Cooking" section about the branch pb/merge-autostash-more on the Git mailing list:

The local changes stashed by "git merge --autostash" were lost when
the merge failed in certain ways, which has been corrected.

Will merge to 'master'.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 2, 2021

This patch series was integrated into seen via git@c3cabae.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

This patch series was integrated into seen via git@45a3338.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

This patch series was integrated into seen via git@3e822dd.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

This patch series was integrated into seen via git@863db97.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

There was a status update in the "Cooking" section about the branch pb/merge-autostash-more on the Git mailing list:

The local changes stashed by "git merge --autostash" were lost when
the merge failed in certain ways, which has been corrected.

Will merge to 'master'.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

This patch series was integrated into seen via git@5fef3b1.

@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

This patch series was integrated into master via git@5fef3b1.

@gitgitgadget gitgitgadget bot added the master label Aug 4, 2021
@gitgitgadget gitgitgadget bot closed this Aug 4, 2021
@gitgitgadget
Copy link

gitgitgadget bot commented Aug 4, 2021

Closed via 5fef3b1.

@phil-blain phil-blain deleted the merge-autostash-ff-fails branch August 17, 2021 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant