New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test-lib-functions: fix test_subcommand_inexact #1185
test-lib-functions: fix test_subcommand_inexact #1185
Conversation
/submit |
Submitted as pull.1185.git.1647894845421.gitgitgadget@gmail.com To fetch this version into
To fetch this version to local tag
|
This branch is now known as |
This patch series was integrated into seen via git@2823f7b. |
On the Git mailing list, Derrick Stolee wrote (reply to this):
|
On the Git mailing list, Junio C Hamano wrote (reply to this):
|
On the Git mailing list, Derrick Stolee wrote (reply to this):
|
On the Git mailing list, Taylor Blau wrote (reply to this):
|
User |
On the Git mailing list, Junio C Hamano wrote (reply to this):
|
On the Git mailing list, Derrick Stolee wrote (reply to this):
|
5017672
to
a3d188c
Compare
On the Git mailing list, Taylor Blau wrote (reply to this):
|
b8819d8
to
ed67b74
Compare
On the Git mailing list, Junio C Hamano wrote (reply to this):
|
On the Git mailing list, Derrick Stolee wrote (reply to this):
|
On the Git mailing list, Abhradeep Chakraborty wrote (reply to this):
|
/submit |
Submitted as pull.1185.v2.git.1648146897.gitgitgadget@gmail.com To fetch this version into
To fetch this version to local tag
|
t/test-lib-functions.sh
Outdated
@@ -1811,8 +1811,8 @@ test_subcommand_inexact () { | |||
shift |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Taylor Blau wrote (reply to this):
On Thu, Mar 24, 2022 at 06:34:57PM +0000, Derrick Stolee via GitGitGadget wrote:
> All existing tests continue to pass with this change. There was one
> instance from t7700-repack.sh that was taking advantage of this
> flexibility, but it was removed in the previous change.
In my tree, the test modified by the previous patch was the only caller
of `test_subcommand_inexact()`. Looking through the output of:
git for-each-ref refs/remotes/origin/ --format='%(refname)' | xargs -L1 \
git -P grep -l test_subcommand_inexact -- t | sort -u
it doesn't look like there are any other topics in flight [1] that call
test_subcommand_inexact(), either.
Unless you have any pending series which want to call
test_subcommand_inexact, I'd be just as happy to get rid of the function
entirely.
Thanks,
Taylor
[1]: My `origin` points to Junio's tree, so I'm looking through all of
the topic branches before integration, not just the standard "master",
"next", etc.
@@ -369,10 +369,56 @@ test_expect_success '--write-midx with preferred bitmap tips' ' | |||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Taylor Blau wrote (reply to this):
On Thu, Mar 24, 2022 at 06:34:56PM +0000, Derrick Stolee via GitGitGadget wrote:
> From: Derrick Stolee <derrickstolee@github.com>
>
> The '--write-midx -b packs non-kept objects' test in t7700-repack.sh
> uses test_subcommand_inexact to check that 'git repack' properly adds
> the '--honor-pack-keep' flag to the 'git pack-objects' subcommand.
> However, the test_subcommand_inexact helper is more flexible than
> initially designed, and this instance is the only one that makes use of
> it: there are additional arguments between 'git pack-objects' and the
> '--honor-pack-keep' flag. In order to make test_subcommand_inexact more
> strict, we need to fix this instance.
>
> This test checks that 'git repack --write-midx -a -b -d' will create a
> new pack-file that does not contain the objects within the kept pack.
> This behavior is possible because of the multi-pack-index bitmap that
> will bitmap objects against multiple packs. Without --write-midx, the
> objects in the kept pack would be duplicated so the resulting pack is
> closed under reachability and bitmaps can be created against it. This is
> discussed in more detail in e4d0c11c0 (repack: respect kept objects with
> '--write-midx -b', 2021-12-20) which also introduced this instance of
> test_subcommand_inexact.
>
> To better verify the intended post-conditions while also removing this
> instance of test_subcommand_inexact, rewrite the test to check the list
> of packed objects in the kept pack and the list of the objects in the
> newly-repacked pack-file _other_ than the kept pack. These lists should
> be disjoint.
>
> Be sure to include a non-kept pack-file and loose objects to be extra
> careful that this is properly behaving with kept packs and not just
> avoiding repacking all pack-files.
Nicely put.
> Co-authored-by: Taylor Blau <me@ttaylorr.com>
> Signed-off-by: Taylor Blau <me@ttaylorr.com>
And to be clear, I am totally OK with this usage of my signed-off-by.
> diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
> index 770d1432046..73452e23896 100755
> --- a/t/t7700-repack.sh
> +++ b/t/t7700-repack.sh
> @@ -369,10 +369,56 @@ test_expect_success '--write-midx with preferred bitmap tips' '
> )
> '
>
> +get_sorted_objects_from_packs () {
> + git show-index <$(cat) >raw &&
It seems a little odd to me to pass the name of a single file as input
to get_sorted_objects_from_packs over stdin. I probably would have
expected something like `git show-index <"$1" >raw && ...` instead.
We may also want to s/packs/pack, since this function only will handle
one index at a time.
> + cut -d" " -f2 raw | sort
Having the sort in there is my fault, but after reading this more
carefully it's definitely unnecessary, since show-index will give us
the results in lexical order by object name already.
> +}
> +
> test_expect_success '--write-midx -b packs non-kept objects' '
> - GIT_TRACE2_EVENT="$(pwd)/trace.txt" \
> - git repack --write-midx -a -b &&
> - test_subcommand_inexact git pack-objects --honor-pack-keep <trace.txt
> + git init repo &&
> + test_when_finished "rm -fr repo" &&
> + (
> + cd repo &&
> +
> + # Create a kept pack-file
> + test_commit base &&
> + git repack -ad &&
> + find $objdir/pack -name "*.idx" >before &&
I thought that here it might be easier to say:
before="$(find $objdir/pack -name "*.idx")"
> + >$objdir/pack/$(basename $(cat before) .idx).keep &&
...and then replace "$(cat before)" with "$before", along with the
other uses of the before file below. But it gets a little funny when
you want to discover which is the new pack, where it is more natural to
dump the output of comm into a file.
> + # Get object list from the one non-kept pack-file
> + comm -13 before after >new-pack &&
You could write "new_pack=$(comm -13 before after)", but debugging this
test would be difficult if the output of comm there contained more than
one line.
> + get_sorted_objects_from_packs \
> + <new-pack \
Though we probably want to check that we only get one line anyway here,
since get_sorted_objects_from_packs will barf if we had more than one
line in file new-pack here, too.
Thanks,
Taylor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Derrick Stolee wrote (reply to this):
On 3/24/2022 2:58 PM, Taylor Blau wrote:
> On Thu, Mar 24, 2022 at 06:34:56PM +0000, Derrick Stolee via GitGitGadget wrote:
>> From: Derrick Stolee <derrickstolee@github.com>
>> diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
>> index 770d1432046..73452e23896 100755
>> --- a/t/t7700-repack.sh
>> +++ b/t/t7700-repack.sh
>> @@ -369,10 +369,56 @@ test_expect_success '--write-midx with preferred bitmap tips' '
>> )
>> '
>>
>> +get_sorted_objects_from_packs () {
>> + git show-index <$(cat) >raw &&
>
> It seems a little odd to me to pass the name of a single file as input
> to get_sorted_objects_from_packs over stdin. I probably would have
> expected something like `git show-index <"$1" >raw && ...` instead.
Based on the way we are creating a file whose contents is the name
of the .idx file, we would at least use '$(cat "$1")'. I kind of like
the symmetry of the input/output redirection when using the helper, but
I can easily change this.
> We may also want to s/packs/pack, since this function only will handle
> one index at a time.
Yes.
>> + cut -d" " -f2 raw | sort
>
> Having the sort in there is my fault, but after reading this more
> carefully it's definitely unnecessary, since show-index will give us
> the results in lexical order by object name already.
Cool. Will drop.
>> +}
>> +
>> test_expect_success '--write-midx -b packs non-kept objects' '
>> - GIT_TRACE2_EVENT="$(pwd)/trace.txt" \
>> - git repack --write-midx -a -b &&
>> - test_subcommand_inexact git pack-objects --honor-pack-keep <trace.txt
>> + git init repo &&
>> + test_when_finished "rm -fr repo" &&
>> + (
>> + cd repo &&
>> +
>> + # Create a kept pack-file
>> + test_commit base &&
>> + git repack -ad &&
>> + find $objdir/pack -name "*.idx" >before &&
>
> I thought that here it might be easier to say:
>
> before="$(find $objdir/pack -name "*.idx")"
>
>> + >$objdir/pack/$(basename $(cat before) .idx).keep &&
>
> ...and then replace "$(cat before)" with "$before", along with the
> other uses of the before file below. But it gets a little funny when
> you want to discover which is the new pack, where it is more natural to
> dump the output of comm into a file.
For this reason, I'll continue to store the .idx names in files.
>> + # Get object list from the one non-kept pack-file
>> + comm -13 before after >new-pack &&
>
> You could write "new_pack=$(comm -13 before after)", but debugging this
> test would be difficult if the output of comm there contained more than
> one line.
>
>> + get_sorted_objects_from_packs \
>> + <new-pack \
>
> Though we probably want to check that we only get one line anyway here,
> since get_sorted_objects_from_packs will barf if we had more than one
> line in file new-pack here, too.
Thanks. Easy to add a test_line_count before this check.
-Stolee
t/test-lib-functions.sh
Outdated
@@ -1811,8 +1811,8 @@ test_subcommand_inexact () { | |||
shift |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Junio C Hamano wrote (reply to this):
"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Derrick Stolee <derrickstolee@github.com>
>
> The implementation of test_subcommand_inexact() was originally
> introduced in e4d0c11c0 (repack: respect kept objects with '--write-midx
> -b', 2021-12-20) with the intention to allow finding a subcommand based
> on an initial set of arguments. The inexactness was intended as a way to
> allow flexible options beyond that initial set, as opposed to
> test_subcommand() which requires that the full list of options is
> provided in its entirety.
>
> The implementation began by copying test_subcommand() and replaced the
> repeated argument 'printf' statement to append ".*" instead of "," to
> each argument. This has a few drawbacks:
>
> 1. Most importantly, this repeats the use of ".*" within 'expr', so the
> inexact match is even more flexible than expected. It allows the list
> of arguments to exist as a subsequence (with any other items included
> between those arguments).
>
> 2. The line 'expr="$(expr%,}"' that previously removed a trailing comma
> now no longer does anything, since the string ends with ".*".
>
> Both of these issues are fixed by keeping the addition of the comma in
> the printf statement, then adding ".*" after stripping out the trailing
> comma.
>
> All existing tests continue to pass with this change. There was one
> instance from t7700-repack.sh that was taking advantage of this
> flexibility, but it was removed in the previous change.
Of course all existing tests continue to pass, as we no longer have
any user of test_subcommand_inexact after the previous step ;-).
Among
(1) doing nothing,
(2) removing, and
(3) clarifying the implementation,
my preference would be 2 > 1 > 3. If we add
(4) clarify the implementation and document what kind of inexactness we
tolerate with an updated comment"
to the mix, that would come before all 3 others, though.
Perhaps squash something like this in?
t/test-lib-functions.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git i/t/test-lib-functions.sh w/t/test-lib-functions.sh
index 0f439c99d6..6f6afae847 100644
--- i/t/test-lib-functions.sh
+++ w/t/test-lib-functions.sh
@@ -1789,8 +1789,8 @@ test_subcommand () {
}
# Check that the given command was invoked as part of the
-# trace2-format trace on stdin, but without an exact set of
-# arguments.
+# trace2-format trace on stdin, but only require that the
+# initial arguments are given as specified.
#
# test_subcommand [!] <command> <args>... < <trace>
#
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Derrick Stolee wrote (reply to this):
On 3/24/2022 4:48 PM, Junio C Hamano wrote:
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> All existing tests continue to pass with this change. There was one
>> instance from t7700-repack.sh that was taking advantage of this
>> flexibility, but it was removed in the previous change.
>
> Of course all existing tests continue to pass, as we no longer have
> any user of test_subcommand_inexact after the previous step ;-).
Yeah, I definitely should have checked to see if there were other
uses of this. I thought there was, but I was mistaken.
> Among
>
> (1) doing nothing,
> (2) removing, and
> (3) clarifying the implementation,
>
> my preference would be 2 > 1 > 3. If we add
I agree that (2) is the best option here.
> (4) clarify the implementation and document what kind of inexactness we
> tolerate with an updated comment"
>
> to the mix, that would come before all 3 others, though.
Is there value in fixing the implementation and adding this comment
if we are to just delete the helper? I suppose that we could prevent
a future contribution from reintroducing the broken implementation.
> Perhaps squash something like this in?
>
> t/test-lib-functions.sh | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git i/t/test-lib-functions.sh w/t/test-lib-functions.sh
> index 0f439c99d6..6f6afae847 100644
> --- i/t/test-lib-functions.sh
> +++ w/t/test-lib-functions.sh
> @@ -1789,8 +1789,8 @@ test_subcommand () {
> }
>
> # Check that the given command was invoked as part of the
> -# trace2-format trace on stdin, but without an exact set of
> -# arguments.
> +# trace2-format trace on stdin, but only require that the
> +# initial arguments are given as specified.
This is an accurate description of what the fixed implementation
does.
My current feeling is that we should just delete this and refer
to that deletion if anyone considers needing something like it.
Thanks,
-Stolee
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the Git mailing list, Junio C Hamano wrote (reply to this):
Derrick Stolee <derrickstolee@github.com> writes:
> On 3/24/2022 4:48 PM, Junio C Hamano wrote:
>> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>>> All existing tests continue to pass with this change. There was one
>>> instance from t7700-repack.sh that was taking advantage of this
>>> flexibility, but it was removed in the previous change.
>>
>> Of course all existing tests continue to pass, as we no longer have
>> any user of test_subcommand_inexact after the previous step ;-).
>
> Yeah, I definitely should have checked to see if there were other
> uses of this. I thought there was, but I was mistaken.
>
>> Among
>>
>> (1) doing nothing,
>> (2) removing, and
>> (3) clarifying the implementation,
>>
>> my preference would be 2 > 1 > 3. If we add
>
> I agree that (2) is the best option here.
>
>> (4) clarify the implementation and document what kind of inexactness we
>> tolerate with an updated comment"
>>
>> to the mix, that would come before all 3 others, though.
>
> Is there value in fixing the implementation and adding this comment
> if we are to just delete the helper? I suppose that we could prevent
> a future contribution from reintroducing the broken implementation.
That is a good thoguth to take into account.
> My current feeling is that we should just delete this and refer
> to that deletion if anyone considers needing something like it.
I am very much in favor of deleting it.
Thanks.
This branch is now known as |
This patch series was integrated into seen via git@efcf525. |
On the Git mailing list, Junio C Hamano wrote (reply to this):
|
This patch series was integrated into seen via git@0d6d19f. |
On the Git mailing list, Abhradeep Chakraborty wrote (reply to this):
|
ed67b74
to
ab0db67
Compare
The '--write-midx -b packs non-kept objects' test in t7700-repack.sh uses test_subcommand_inexact to check that 'git repack' properly adds the '--honor-pack-keep' flag to the 'git pack-objects' subcommand. However, the test_subcommand_inexact helper is more flexible than initially designed, and this instance is the only one that makes use of it: there are additional arguments between 'git pack-objects' and the '--honor-pack-keep' flag. In order to make test_subcommand_inexact more strict, we need to fix this instance. This test checks that 'git repack --write-midx -a -b -d' will create a new pack-file that does not contain the objects within the kept pack. This behavior is possible because of the multi-pack-index bitmap that will bitmap objects against multiple packs. Without --write-midx, the objects in the kept pack would be duplicated so the resulting pack is closed under reachability and bitmaps can be created against it. This is discussed in more detail in e4d0c11 (repack: respect kept objects with '--write-midx -b', 2021-12-20) which also introduced this instance of test_subcommand_inexact. To better verify the intended post-conditions while also removing this instance of test_subcommand_inexact, rewrite the test to check the list of packed objects in the kept pack and the list of the objects in the newly-repacked pack-file _other_ than the kept pack. These lists should be disjoint. Be sure to include a non-kept pack-file and loose objects to be extra careful that this is properly behaving with kept packs and not just avoiding repacking all pack-files. Co-authored-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
The implementation of test_subcommand_inexact() was originally introduced in e4d0c11 (repack: respect kept objects with '--write-midx -b', 2021-12-20) with the intention to allow finding a subcommand based on an initial set of arguments. The inexactness was intended as a way to allow flexible options beyond that initial set, as opposed to test_subcommand() which requires that the full list of options is provided in its entirety. The implementation began by copying test_subcommand() and replaced the repeated argument 'printf' statement to append ".*" instead of "," to each argument. This caused it to be more flexible than initially intended. The previous change deleted the only use of test_subcommand_inexact, so instead of editing the helper, delete it. Helped-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
ab0db67
to
f5a96a1
Compare
/submit |
Submitted as pull.1185.v3.git.1648234967.gitgitgadget@gmail.com To fetch this version into
To fetch this version to local tag
|
This patch series was integrated into seen via git@5ff6174. |
This patch series was integrated into seen via git@aa6d166. |
This patch series was integrated into seen via git@22cb3c2. |
There was a status update in the "New Topics" section about the branch Test clean-up. Will merge to 'next'. source: <pull.1185.v3.git.1648234967.gitgitgadget@gmail.com> |
This patch series was integrated into seen via git@b193d33. |
This patch series was integrated into next via git@fb03a86. |
On the Git mailing list, Taylor Blau wrote (reply to this):
|
Junio discovered in [1] that test_subcommand_inexact is more flexible than initially intended.
[1] https://lore.kernel.org/git/xmqq4k41vdwe.fsf@gitster.g/
The intention was that we do not need to specify the remaining arguments for a subcommand, but instead the current behavior is to allow the given arguments to appear as any subsequence within the command (except that the first "git" instance must be the first argument).
By changing the test that needed the helper, we can avoid the helper in the first place. Modify the test and remove the helper.
Changes in v3
Significant edits to the test in t7700 based on Junio and Taylor's feedback.
Patch 2 now deletes the helper as it is not used anywhere.
Thanks,
-Stolee
cc: gitster@pobox.com
cc: chakrabortyabhradeep79@gmail.com
cc: Taylor Blau me@ttaylorr.com