Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sparse-checkout set crashes #607

Merged
merged 3 commits into from
Sep 20, 2023
Merged

Fix sparse-checkout set crashes #607

merged 3 commits into from
Sep 20, 2023

Conversation

vdye
Copy link
Collaborator

@vdye vdye commented Sep 20, 2023

Thanks for taking the time to contribute to Git!

This fork contains changes specific to monorepo scenarios. If you are an
external contributor, then please detail your reason for submitting to
this fork:

  • This is an early version of work already under review upstream.
  • This change only applies to interactions with Azure DevOps and the
    GVFS Protocol.
  • This change only applies to the virtualization hook and VFS for Git.

This PR corrects a couple of issues in order to fix #606:

  • Commit 1 ensures the dtype arg to path_matches_pattern_list() is always initialized, since the code added in 2cff5e9 requires an initialized value.
  • Commit 2 fixes copying & modification of the index name_hash, dir_hash, and name_hash_initialized in expand_index(). This problem originates upstream (and should eventually be fixed there as well), but it doesn't actually trigger any bugs there; it's the interaction with 2cff5e9 that causes the segfault in our fork.
  • Commit 3 adds a test around the treatment of untracked files & directories when sparse-checkout patterns are updated, moving them from "in cone" to "outside of cone." Again, while this is a good thing to have upstream (I plan to submit it eventually), its real value here is showing that this segfault no longer occurs.

The 'dtype' arg to 'path_matches_pattern_list()' is not assumed to be
initialized, and in a few callers it is not. However, the changes in
2cff5e9 (Add virtual file system settings and hook proc, 2018-01-11)
_do_ assume 'dtype' is initialized.

This can cause indeterminate behavior. For example, seemingly unrelated
changes from 33b1b4c (sparse-checkout: avoid using internal API of
unpack-trees, take 2, 2023-02-27) changed the initial value of 'dtype' in
'expand_index()' from non-zero to zero. That caused
'path_matches_pattern_list()' to call 'resolve_dtype()', which initialized
the 'name_hash' and 'dir_hash' of 'istate', which triggered another bug that
ultimately led to a segfault.

Update the callers with uninitialized 'dtype' values to intialize with a
value appropriate to their respective use cases.
In ac8acb4 (sparse-index: complete partial expansion, 2022-05-23),
'expand_index()' was updated to expand the index to a given pathspec.
However, the 'path_matches_pattern_list()' method used to facilitate this
has the side effect of initializing or updating the index hash variables
('name_hash', 'dir_hash', and 'name_hash_initialized'). This operation is
performed on 'istate', though, not 'full'; as a result, the initialized
hashes are later overwritten when copied from 'full'. To ensure the correct
hashes are in 'istate' after the index expansion, change the arg used in
'path_matches_pattern_list()' from 'istate' to 'full'.

Note that this does not fully solve the problem. If 'istate' does not have
an initialized 'name_hash' when its contents are copied to 'full',
initialized hashes will be copied back into 'istate' but
'name_hash_initialized' will be 0. Therefore, we also need to copy
'full->name_hash_initialized' back to 'istate' after the index expansion is
complete.

Signed-off-by: Victoria Dye <vdye@github.com>
Add a test verifying that sparse-checkout (with and without sparse index
enabled) treat untracked files & directories correctly when changing sparse
patterns. Specifically, it ensures that 'git sparse-checkout set'

* deletes empty directories outside the sparse cone
* does _not_ delete untracked files outside the sparse cone

Signed-off-by: Victoria Dye <vdye@github.com>
@vdye vdye self-assigned this Sep 20, 2023
@vdye vdye marked this pull request as ready for review September 20, 2023 20:30
Copy link
Collaborator

@jeffhostetler jeffhostetler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

Good find on the very obscure dtype init.

Copy link
Member

@mjcheetham mjcheetham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@vdye vdye merged commit 00315c6 into microsoft:vfs-2.42.0 Sep 20, 2023
43 of 44 checks passed
@vdye vdye deleted the vdye/sparse-checkout-crash-fix branch September 20, 2023 21:21
@aaronberger-msft
Copy link

Thank you guys so much! Awesome work and super fast response! Kudos!

dscho pushed a commit that referenced this pull request Nov 3, 2023
dscho pushed a commit that referenced this pull request Nov 3, 2023
dscho pushed a commit that referenced this pull request Nov 3, 2023
dscho pushed a commit that referenced this pull request Nov 3, 2023
dscho pushed a commit that referenced this pull request Nov 8, 2023
dscho pushed a commit that referenced this pull request Nov 14, 2023
dscho pushed a commit that referenced this pull request Nov 20, 2023
vdye added a commit that referenced this pull request Feb 27, 2024
dscho pushed a commit that referenced this pull request Apr 23, 2024
dscho pushed a commit that referenced this pull request Apr 23, 2024
dscho pushed a commit that referenced this pull request Apr 23, 2024
dscho pushed a commit that referenced this pull request Apr 23, 2024
dscho pushed a commit that referenced this pull request Apr 24, 2024
dscho pushed a commit that referenced this pull request Apr 29, 2024
dscho pushed a commit that referenced this pull request May 14, 2024
dscho pushed a commit that referenced this pull request May 14, 2024
dscho pushed a commit that referenced this pull request May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crashing during sparse-checkout operations
4 participants