Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mergetool(vimdiff): allow paths to contain spaces again #1287

Closed

Conversation

dscho
Copy link
Member

@dscho dscho commented Jul 13, 2022

In git-for-windows#3945, it was reported that as of v2.37.0, git mergetool --tool=vimdiff fails to handle paths containing spaces. Let's fix that.

Changes since v1:

  • Using base_present=false instead of the misleading base_present=1 (thanks Junio & Fernando!)

Cc: Fernando Ramos greenfoo@u92.eu

@dscho
Copy link
Member Author

dscho commented Jul 13, 2022

/submit

@dscho dscho self-assigned this Jul 13, 2022
@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

Submitted as pull.1287.git.1657726969774.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v1

To fetch this version to local tag pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v1:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v1

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> To fix this, let's not expand the variables containing the path
> parameters before passing them to the `eval` command, but let that
> command expand the variables instead.

Yup, that is exactly the right approach to fix this kind of
breakage.

Thanks for a clear description of the problem and the solution.

Fernando?  Ack?

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

This branch is now known as js/vimdiff-quotepath-fix.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

This patch series was integrated into seen via git@c9a0646.

@gitgitgadget gitgitgadget bot added the seen label Jul 13, 2022
@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

This patch series was integrated into seen via git@7b49f7b.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Fernando Ramos wrote (reply to this):

On 22/07/13 09:22AM, Junio C Hamano wrote:
> Thanks for a clear description of the problem and the solution.
> 
> Fernando?  Ack?

Yes. 100% Ack'ed.

Thanks Johannes for the bug report and the patch!

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

Fernando Ramos <greenfoo@u92.eu> writes:

> On 22/07/13 09:22AM, Junio C Hamano wrote:
>> Thanks for a clear description of the problem and the solution.
>> 
>> Fernando?  Ack?
>
> Yes. 100% Ack'ed.
>
> Thanks Johannes for the bug report and the patch!

Funny that we now seem to fail t7609 mergetool --tool=vimdiff
tests, e.g.

    https://github.com/git/git/actions/runs/2665800752

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

>  	if $base_present
>  	then
> ...
> +	base_present=1

I think s/1/true/ or something is in order, perhaps?

> +	LOCAL='lo cal'
> +	BASE='ba se'
> +	REMOTE="' '"
> +	MERGED='mer ged'
> +	merge_tool_path=record_parameters
> +
> +	merge_cmd vimdiff || at_least_one_ko=true
> +
> +	cat >expect <<-\EOF
> +	-f
> +	-c
> +	echo | split | vertical split | 1b | wincmd l | vertical split | quit | wincmd l | 2b | wincmd j | 3b | tabdo windo diffthis
> +	-c
> +	tabfirst
> +	lo cal
> +	' '
> +	mer ged
> +	EOF
> +
> +	diff -u expect actual || at_least_one_ko=true
> +
>  	if test "$at_least_one_ko" = "true"
>  	then
>  		return 255
>
> base-commit: 980145f7470e20826ca22d7343494712eda9c81d

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Fernando Ramos wrote (reply to this):

On 22/07/13 02:08PM, Junio C Hamano wrote:
> 
> I think s/1/true/ or something is in order, perhaps?
> 

Yes. I was just looking into that.

For one, as you said, "1" should be "true". That also changes the expected
output.

Then, in addition, the expected output needs to be re-adjusted once again if we
plan to apply this patch on top of the other one from two days ago (the one that
adds the "leftabove" keyword to split subcommands).

After these changes, this is how the original patch from Johannes needs to be
updated:


diff --git a/mergetools/vimdiff b/mergetools/vimdiff
index 56516ae271..3046dcd0dc 100644
--- a/mergetools/vimdiff
+++ b/mergetools/vimdiff
@@ -623,7 +623,7 @@ run_unit_tests () {
 		done
 	}
 
-	base_present=1
+	base_present=true
 	LOCAL='lo cal'
 	BASE='ba se'
 	REMOTE="' '"
@@ -635,10 +635,11 @@ run_unit_tests () {
 	cat >expect <<-\EOF
 	-f
 	-c
-	echo | split | vertical split | 1b | wincmd l | vertical split | quit | wincmd l | 2b | wincmd j | 3b | tabdo windo diffthis
+	echo | -tabnew | leftabove split | leftabove vertical split | 1b | wincmd l | leftabove vertical split | 2b | wincmd l | 3b | wincmd j | 4b | tabnext | -tabnew | leftabove vertical split | 2b | wincmd l | 1b | tabnext | -tabnew | leftabove vertical split | 2b | wincmd l | 3b | tabnext | leftabove vertical split | leftabove split | 1b | wincmd j | leftabove split | 2b | wincmd j | 3b | wincmd l | 4b | tabdo windo diffthis
 	-c
 	tabfirst
 	lo cal
+	ba se
 	' '
 	mer ged
 	EOF
-- 
2.37.0

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

Fernando Ramos <greenfoo@u92.eu> writes:

> On 22/07/13 02:08PM, Junio C Hamano wrote:
>> 
>> I think s/1/true/ or something is in order, perhaps?
>> 
>
> Yes. I was just looking into that.
>
> For one, as you said, "1" should be "true". That also changes the expected
> output.

OK, because "1" fails to execute, the expected output Dscho had
(which is for the case without base) would become invalid when we
use "true".

Perhaps we should use "false" instead?  Or do we need to test both?

> Then, in addition, the expected output needs to be re-adjusted once again if we
> plan to apply this patch on top of the other one from two days ago (the one that
> adds the "leftabove" keyword to split subcommands).

> After these changes, this is how the original patch from Johannes needs to be
> updated:
>
>
> diff --git a/mergetools/vimdiff b/mergetools/vimdiff
> index 56516ae271..3046dcd0dc 100644
> --- a/mergetools/vimdiff
> +++ b/mergetools/vimdiff
> @@ -623,7 +623,7 @@ run_unit_tests () {
>  		done
>  	}
>  
> -	base_present=1
> +	base_present=true
>  	LOCAL='lo cal'
>  	BASE='ba se'
>  	REMOTE="' '"
> @@ -635,10 +635,11 @@ run_unit_tests () {
>  	cat >expect <<-\EOF
>  	-f
>  	-c
> -	echo | split | vertical split | 1b | wincmd l | vertical split | quit | wincmd l | 2b | wincmd j | 3b | tabdo windo diffthis
> +	echo | -tabnew | leftabove split | leftabove vertical split | 1b | wincmd l | leftabove vertical split | 2b | wincmd l | 3b | wincmd j | 4b | tabnext | -tabnew | leftabove vertical split | 2b | wincmd l | 1b | tabnext | -tabnew | leftabove vertical split | 2b | wincmd l | 3b | tabnext | leftabove vertical split | leftabove split | 1b | wincmd j | leftabove split | 2b | wincmd j | 3b | wincmd l | 4b | tabdo windo diffthis
>  	-c
>  	tabfirst
>  	lo cal
> +	ba se
>  	' '
>  	mer ged
>  	EOF

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

Junio C Hamano <gitster@pobox.com> writes:

>> For one, as you said, "1" should be "true". That also changes the expected
>> output.
>
> OK, because "1" fails to execute, the expected output Dscho had
> (which is for the case without base) would become invalid when we
> use "true".
>
> Perhaps we should use "false" instead?  Or do we need to test both?

How confident are you with the "leftabove" stuff?  Is it solid
enough that it is safe to assume that we can queue this fix on top
of it?

This is how it would look like on top of the "leftabove" topic.

 mergetools/vimdiff | 39 +++++++++++++++++++++++++++++++++++----
 1 file changed, 35 insertions(+), 4 deletions(-)

diff --git c/mergetools/vimdiff w/mergetools/vimdiff
index b045b10fd7..f770b8fe24 100644
--- c/mergetools/vimdiff
+++ w/mergetools/vimdiff
@@ -414,8 +414,8 @@ merge_cmd () {
 
 	if $base_present
 	then
-		eval "$merge_tool_path" \
-			-f "$FINAL_CMD" "$LOCAL" "$BASE" "$REMOTE" "$MERGED"
+		eval '"$merge_tool_path"' \
+			-f "$FINAL_CMD" '"$LOCAL"' '"$BASE"' '"$REMOTE"' '"$MERGED"'
 	else
 		# If there is no BASE (example: a merge conflict in a new file
 		# with the same name created in both braches which didn't exist
@@ -424,8 +424,8 @@ merge_cmd () {
 		FINAL_CMD=$(echo "$FINAL_CMD" | \
 			sed -e 's:2b:quit:g' -e 's:3b:2b:g' -e 's:4b:3b:g')
 
-		eval "$merge_tool_path" \
-			-f "$FINAL_CMD" "$LOCAL" "$REMOTE" "$MERGED"
+		eval '"$merge_tool_path"' \
+			-f "$FINAL_CMD" '"$LOCAL"' '"$REMOTE"' '"$MERGED"'
 	fi
 
 	ret="$?"
@@ -614,6 +614,37 @@ run_unit_tests () {
 		fi
 	done
 
+	# verify that `merge_cmd` handles paths with spaces
+	record_parameters () {
+		>actual
+		for arg
+		do
+			echo "$arg" >>actual
+		done
+	}
+
+	base_present=false
+	LOCAL='lo cal'
+	BASE='ba se'
+	REMOTE="' '"
+	MERGED='mer ged'
+	merge_tool_path=record_parameters
+
+	merge_cmd vimdiff || at_least_one_ko=true
+
+	cat >expect <<-\EOF
+	-f
+	-c
+	echo | leftabove split | leftabove vertical split | 1b | wincmd l | leftabove vertical split | quit | wincmd l | 2b | wincmd j | 3b | tabdo windo diffthis
+	-c
+	tabfirst
+	lo cal
+	' '
+	mer ged
+	EOF
+
+	diff -u expect actual || at_least_one_ko=true
+
 	if test "$at_least_one_ko" = "true"
 	then
 		return 255

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Fernando Ramos wrote (reply to this):

On 22/07/13 02:54PM, Junio C Hamano wrote:
> 
> OK, because "1" fails to execute, the expected output Dscho had
> (which is for the case without base) would become invalid when we
> use "true".
> 
> Perhaps we should use "false" instead?  Or do we need to test both?
> 
I think testing both is not really needed as the unit test is just making sure
that filenames with spaces are processed correctly.  Whatever comes before
(which changes depending on the value of "base_present") doesn't really matter
as long as there is something.

If we have to select one ("true" or "false") using "true" seems more practical,
as that way the BASE file is also included, and thus an extra argument is
tested.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Fernando Ramos wrote (reply to this):

On 22/07/13 03:06PM, Junio C Hamano wrote:
>
> How confident are you with the "leftabove" stuff?  Is it solid
> enough that it is safe to assume that we can queue this fix on top
> of it?
> 
I would say yes. I have been testing it since without problems.

But it is also true that the "leftabove" patch fixes a corner case that can wait
until the next release.

I will keep testing it in any case and let you know if I find a case where it
doesn't work.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

This patch series was integrated into seen via git@530622f.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 13, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

Fernando Ramos <greenfoo@u92.eu> writes:

> On 22/07/13 02:54PM, Junio C Hamano wrote:
>> 
>> OK, because "1" fails to execute, the expected output Dscho had
>> (which is for the case without base) would become invalid when we
>> use "true".
>> 
>> Perhaps we should use "false" instead?  Or do we need to test both?
>> 
> I think testing both is not really needed as the unit test is just making sure
> that filenames with spaces are processed correctly.  Whatever comes before
> (which changes depending on the value of "base_present") doesn't really matter
> as long as there is something.

As the quoting glitch are distributed on both sides

	if $base_present
	then
		eval "$merge_tool_path" \
			-f "$FINAL_CMD" "$LOCAL" "$BASE" "$REMOTE" "$MERGED"
	else
		# If there is no BASE (example: a merge conflict in a new file
		# with the same name created in both braches which didn't exist
		# before), close all BASE windows using vim's "quit" command

		FINAL_CMD=$(echo "$FINAL_CMD" | \
			sed -e 's:2b:quit:g' -e 's:3b:2b:g' -e 's:4b:3b:g')

		eval "$merge_tool_path" \
			-f "$FINAL_CMD" "$LOCAL" "$REMOTE" "$MERGED"
	fi

I suspect you can fix only one, and carefully choose which side to
test, and leave the other side broken ;-)

In any case, to minimize disruption to Dscho's patch, I replaced the
"1" with "false" (which will make his patch pass the new test
without other stuff) and tweak its merge into 'seen' to adjust for
the "leftabove" topic.

Thanks.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 14, 2022

There was a status update in the "New Topics" section about the branch js/vimdiff-quotepath-fix on the Git mailing list:

Variable quoting fix in the vimdiff driver of "git mergetool"

Expecting a reroll.
cf. <xmqqa69cabhq.fsf@gitster.g>
source: <pull.1287.git.1657726969774.gitgitgadget@gmail.com>

In 0041797 (vimdiff: new implementation with layout support,
2022-03-30), we introduced a completely new implementation of the
`vimdiff` backend for `git mergetool`.

In this implementation, we no longer call `vim` directly but we
accumulate in the variable `FINAL_CMD` an arbitrary number of commands
for `vim` to execute, which necessitates the use of `eval` to split the
commands properly into multiple command-line arguments.

That same `eval` command also needs to pass the paths to `vim`, and
while it looks as if they are quoted correctly, that quoting only
reaches the `eval` instruction and is lost after that, therefore paths
that contain whitespace characters (or other characters that are
interpreted by the POSIX shell) are handled incorrectly.

This is a simple reproducer:

	git init -b main bam-merge-fail
	cd bam-merge-fail
	echo a>"a file.txt"
	git add "a file.txt"
	git commit -m "added 'a file.txt'"
	echo b>"a file.txt"
	git add "a file.txt"
	git commit -m "diverged b 'a file.txt'"
	git checkout -b c HEAD~
	echo c>"a file.txt"
	git add "a file.txt"
	git commit -m "diverged c 'a file.txt'"
	git checkout main
	git merge c
	git mergetool --tool=vimdiff

With Git v2.37.0/v2.37.1, this will open 7 buffers, not four, and not
display the correct contents at all.

To fix this, let's not expand the variables containing the path
parameters before passing them to the `eval` command, but let that
command expand the variables instead.

This fixes git-for-windows#3945

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho dscho force-pushed the fix-vimdiff-with-spaces-in-paths branch from dde9c7c to 7de381b Compare July 14, 2022 13:07
@dscho
Copy link
Member Author

dscho commented Jul 14, 2022

/submit

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 14, 2022

Submitted as pull.1287.v2.git.1657809063728.gitgitgadget@gmail.com

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v2

To fetch this version to local tag pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v2:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-1287/dscho/fix-vimdiff-with-spaces-in-paths-v2

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 14, 2022

On the Git mailing list, Junio C Hamano wrote (reply to this):

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

>  	if $base_present
>  	then
> -		eval "$merge_tool_path" \
> -			-f "$FINAL_CMD" "$LOCAL" "$BASE" "$REMOTE" "$MERGED"
> +		eval '"$merge_tool_path"' \
> +			-f "$FINAL_CMD" '"$LOCAL"' '"$BASE"' '"$REMOTE"' '"$MERGED"'
>  	else
>  		# If there is no BASE (example: a merge conflict in a new file
>  		# with the same name created in both braches which didn't exist
> @@ -424,8 +424,8 @@ merge_cmd () {
>  		FINAL_CMD=$(echo "$FINAL_CMD" | \
>  			sed -e 's:2b:quit:g' -e 's:3b:2b:g' -e 's:4b:3b:g')
>  
> -		eval "$merge_tool_path" \
> -			-f "$FINAL_CMD" "$LOCAL" "$REMOTE" "$MERGED"
> +		eval '"$merge_tool_path"' \
> +			-f "$FINAL_CMD" '"$LOCAL"' '"$REMOTE"' '"$MERGED"'
>  	fi

If there were another syntactic fix we need in the future, we may by
mistake fix one but not the other, and the test we add in this patch
checks only one side but not the other.  In a follow-up we may want
to unify the two eval invocations to make the testing of this part
more robust.

But as a minimum and obvious fix, stopping at the above is
appropriate for this patch.

> + ...
> +	diff -u expect actual || at_least_one_ko=true

I wonder if we still should care about platforms that need to set
GIT_TEST_CMP_USE_COPIED_CONTEXT while running our tests.  If we use
"diff -u" hardcoded like this here, we are declaring that they are
now unwelcome to run our tests.

It might be just the matter of using "cmp -s" here (or run "diff"
without any option).  Do we care about emitting the difference in
the output?  I doubt it.

>  	if test "$at_least_one_ko" = "true"
>  	then
>  		return 255

Thanks.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 14, 2022

This patch series was integrated into seen via git@145377a.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 15, 2022

This patch series was integrated into seen via git@236050e.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 15, 2022

This patch series was integrated into next via git@4273bbd.

@gitgitgadget gitgitgadget bot added the next label Jul 15, 2022
@gitgitgadget
Copy link

gitgitgadget bot commented Jul 18, 2022

This patch series was integrated into seen via git@8a7545e.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 19, 2022

This patch series was integrated into seen via git@761650e.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 20, 2022

This patch series was integrated into seen via git@cb29110.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 20, 2022

There was a status update in the "Cooking" section about the branch js/vimdiff-quotepath-fix on the Git mailing list:

Variable quoting fix in the vimdiff driver of "git mergetool"

Will merge to 'master'.
source: <pull.1287.v2.git.1657809063728.gitgitgadget@gmail.com>

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 20, 2022

This patch series was integrated into seen via git@83051c9.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 20, 2022

This patch series was integrated into seen via git@d7856b9.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 21, 2022

This patch series was integrated into seen via git@9176609.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 22, 2022

This patch series was integrated into seen via git@4483ea9.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 22, 2022

This patch series was integrated into master via git@4483ea9.

@gitgitgadget
Copy link

gitgitgadget bot commented Jul 22, 2022

This patch series was integrated into next via git@4483ea9.

@gitgitgadget gitgitgadget bot added the master label Jul 22, 2022
@gitgitgadget gitgitgadget bot closed this Jul 22, 2022
@gitgitgadget
Copy link

gitgitgadget bot commented Jul 22, 2022

Closed via 4483ea9.

@dscho dscho deleted the fix-vimdiff-with-spaces-in-paths branch July 25, 2022 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant