worktree stack supports excludes #397

Byron · 2022-04-21T03:28:06Z

A way to obtain a set of attributes given a path, efficiently.

Each path we handle needs one lookup there during checkout, thus
it must be lazy and per thread.

git seems to implement it as stack which is efficient if similar paths are handled together,
and then drops information as it is no longer needed. This seems like the way to go
as we never have to optimize for random access.

Tasks

study existing dircache stack and see if it can be generalized
implement stack for git-ignore and use git-glob for matching and be sure dir-excludes work
gix repo exclude query

It doesn't work as one would expect due to it wanting to match relative paths only. Thus it's better to spare folks the surprise and instead use `wildmatch()` directly. It works the same, but doesn't have certain shortcuts which aren't needed for standard matches anyway.

It caches as much as it can for each pattern list. Many more tests are needed, but that will come in with tests that compare git with our implementation.

… from files (#301)

Also allow to read from bytes directly, which happens if the file is read from the index/odb.

Also it turns out that we can't really unify the pattern list creation yet, unless we'd add parse errors to the trait which maybe is preferable over duplicating some code.

…which haven't been covered by tests and which show that the current implementation isn't sufficient. It also makes clear that the match styles of git-attributes and git-ignore are quite different. It's unclear why, even though the way both algorithms work are quite different. So have to take a step back and find a way to figure it out.

The failing tests are due to shortcomings in git-glob pattern matching right now.

…#301) The stack-based matching is responsible for excluding entire directories which is something the git-attributes based matching doesn't do.

…itignore files (#301)

…301) This makes it easier to work with errors in practice, we are either interested in the value that failed to parse to try something else or want a nice user message. Having one decode error type facilitates that.

This way the original pattern can be reproduced on the fly without actually storing it, saving one allocation.

…but it feels a little like a hack. Maybe this should tell us that we need to reorganize code elsewhere?

)" This reverts commit d0c8407.

Very similar to `string()`, but as path, whose query can never fail.

That way it's possible to call certain methods that take a separate path buffer.

XDG home is still needs to be made accessible, along with a few other improvements.

A lightweight, general purpose error to display permissions violations that cause errors. This should make it useable across crates.

…ety (#301)

…ery` (#301) Details are still to be fixed, and reading from stdin should be implemented one day. Issue right now is that the source path is normalized for some reason, let's see where that happens.

If and only if it canonicalized the source path. That way, users will still get a familiar path. This is due to `parent()` not operating in the file system, which otherwise would be equivalent to `..`, but that's not how we work. Maybe we should overhaul the way this works to use `../` instead and just 'absoluteize' the path later (std::path::absolute()) is on the way for that.

It doesn't reproduce on a local VM though.

…301) Fixes windows CI issue, I hope… .

Byron added 30 commits April 21, 2022 11:25

adapt to changes in git-glob (#301)

0effef0

Basic match group pattern matching (#301)

cc1312d

It caches as much as it can for each pattern list. Many more tests are needed, but that will come in with tests that compare git with our implementation.

Baseline tests for global excludes and instantiation of pattern lists…

afbb295

… from files (#301)

refactor (#301)

0852f13

Also allow to read from bytes directly, which happens if the file is read from the index/odb.

enforce nicer/unified names so use struct instead of tuple (#301)

4c9a51e

Also it turns out that we can't really unify the pattern list creation yet, unless we'd add parse errors to the trait which maybe is preferable over duplicating some code.

first succeding tests for global repository excludes (#301)

4a1e797

thanks clippy

ac53780

more pendantic baseline parsing (#301)

99c7b5f

support for loading per-directory pattern lists as well (#301)

457c921

generalize parsing of paths in pattern lists (#301)

f66c27e

thanks clippy

5992883

try using compatct_str for attribute storage (#301)

50b8c64

feat: publicly accessible Result type (#301)

f1635c3

refactor (#301)

fe9fb4c

adapt to changes in git-glob and add failing test (#301)

cd58a1c

The failing tests are due to shortcomings in git-glob pattern matching right now.

adjust baseline to only handle patterns that work without a dir stack (…

fb65a39

…#301) The stack-based matching is responsible for excluding entire directories which is something the git-attributes based matching doesn't do.

Allow basename matches to work like before (#301)

4f6cefc

cleanup (#301)

1ab4705

also skip negative attribute patterns (#301)

04ab5d3

Sketch how attribute states can be used (#301)

d80b321

sketch how attribute globals could be used in worktrees (#301)

97ee03d

Sketch state for handling excludes (#301)

eb525f7

fix release build

f7c1920

Sketch state for handling attributes as well (#301)

d87d62d

re-export git-glob as its Case type is part of the public API (#301)

4b72045

Add baseline test to motivate implementing ignore file stack (#301)

ce40add

feat: add Default impl for pattern::Case (#301)

2c88b57

refactor to make push/pop with mutable state work; prepare to read .g…

8d1000b

…itignore files (#301)

Byron added 28 commits April 28, 2022 18:57

refactor (#301)

807b7f8

some tests to check pattern negation (#301)

2672a25

feat: fmt::Display impl for Pattern. (#301)

455a72e

This way the original pattern can be reproduced on the fly without actually storing it, saving one allocation.

make use of new git-glob::Pattern::to_string() feature (#301)

d29932d

refactor (#301)

a86ed7b

Turn attribute files into a Cow to support other usecases… (#301)

d0c8407

…but it feels a little like a hack. Maybe this should tell us that we need to reorganize code elsewhere?

Revert "Turn attribute files into a Cow to support other usecases… (#301

ed7f223

)" This reverts commit d0c8407.

feat: GitConfig::path() for direct access to paths. (#301)

7c75eac

Very similar to `string()`, but as path, whose query can never fail.

feat: State::path_backing(). (#301)

8ab219a

That way it's possible to call certain methods that take a separate path buffer.

preliminary access to a fully configured exclusion cache (#301)

259d015

XDG home is still needs to be made accessible, along with a few other improvements.

thanks clippy

5bf6b52

Support for overrides on the command-line (#301)

7d98b21

feat: permission::Error (#301)

de0226a

A lightweight, general purpose error to display permissions violations that cause errors. This should make it useable across crates.

Some notes about of 'path' will soon have to be amended with more saf…

97e53f6

…ety (#301)

feat: A shared permission::Error type (#301)

95577e2

Permission controlled access to xdg config (#301)

42a6c8c

refactor (#301)

a89a667

thanks clippy

f802a03

fix build (#301)

cb1c80f

Basic prefix support as well the first working version of `exclude qu…

9cb8385

…ery` (#301) Details are still to be fixed, and reading from stdin should be implemented one day. Issue right now is that the source path is normalized for some reason, let's see where that happens.

Add --show-ignore-patterns to gix repo exclude query (#301)

09f904b

see if this fixes the CI test issue on windows (#301)

7697f51

It doesn't reproduce on a local VM though.

Allow reading patterns from stdin (#301)

0c597fe

thanks clippy

056e8d2

REMOVE ME: debug info for failing CI test (#301)

b0b3df4

Don't have expectations on the path, rather deal with it gracefully (#…

3a41d5c

…301) Fixes windows CI issue, I hope… .

Byron merged commit 98da8ba into main Apr 30, 2022

Byron deleted the worktree-stack branch April 30, 2022 02:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

worktree stack supports excludes #397

worktree stack supports excludes #397

Byron commented Apr 21, 2022 •

edited

worktree stack supports excludes #397

worktree stack supports excludes #397

Conversation

Byron commented Apr 21, 2022 • edited

Tasks

Byron commented Apr 21, 2022 •

edited