Dont traverse gitignored dirs for gitignore files

[robramsaynz]

Updated version of #397.

this seems to be an upstream issue in go-git, and i'll prepare a PR for them soon, but for now this copies in the affected function and fixes it by checking the accumulated patterns while walking the fs looking for gitignore files

fixes: #389

• is located github.com/ackama/osv-scanner, rather than github.com/robotdana/osv-scanner
• has tests
• has extra changes to handle Don't traverse gitignored dirs for gitignore files #397 (comment) (handles mid-tree dirs, handles non-git dirs, handles recursive flag)

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[robramsaynz]

@another-rex This is ready to look at now. I may add a few more tests to it, but it currently works, and has reasonable test coverage. I'm not sure how to add you as a reviewer so I'm just @-ing you here.

[robramsaynz]

@another-rex RE:

Also, how different is this from upstream? I think ideally we push this change upstream, and avoid having to maintain this. But since it's only a few files, I'm not too concerned if we can't get it upstream.

I've moved things round to make them easier to diff, but it shouldn't be massively different. You can see the difference with:

I've tried to keep these fairly similar to make diffing easier. You can check the diff with:

$ curl https://raw.githubusercontent.com/go-git/go-git/v5.7.0/plumbing/format/gitignore/dir.go > a.go
$ curl https://raw.githubusercontent.com/ackama/osv-scanner/dont-traverse-gitignored-dirs-for-gitignore-files/pkg/osvscanner/customgitignore/dir.go > b.go
$ diff -u a.go b.go

We are messing with a parsing algorithm, so that's risky a little, but also have fairly good tests for this generally. I'm expecting upstream to like the changes. My guess is they wont want walk_up_to_root.go given they're a general purpose library and this is the "make it work like ripgrep" file. For instance the existing API of ReadPatterns() seems to assume it gets passed the root path of a git repo.

[another-rex]

LGTM!

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 71.83908% with 49 lines in your changes are missing coverage. Please review.

Project coverage is 59.86%. Comparing base (b28c1c8) to head (f38dff0).
Report is 3 commits behind head on main.

Files	Patch %	Lines
internal/customgitignore/walk_up_to_root.go	72.13%	24 Missing and 10 partials ⚠️
internal/customgitignore/dir.go	68.75%	12 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #797      +/-   ##
==========================================
+ Coverage   59.78%   59.86%   +0.07%     
==========================================
  Files         136      138       +2     
  Lines       11268    11424     +156     
==========================================
+ Hits         6737     6839     +102     
- Misses       4102     4145      +43     
- Partials      429      440      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[G-Rath]

#832 has been merged so this can be actioned now, though I don't think it should block landing this