Add `--ignore-file` flag #156

alok · 2017-10-31T01:27:40Z

Basically identical to rg's same flag.

The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy. Also, if I'm searching from ~, I don't expect the gitignore file several levels down in some project to get picked up by default.

The text was updated successfully, but these errors were encountered:

sharkdp · 2017-11-01T20:17:40Z

Thank you for the feedback.

Basically identical to rg's same flag.

The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy.

If I understand correctly, rgs flag is used to specify an additional ignore file (with low precedence), so this doesn't solve the use case you descibed(?).

Have you seen fds --exclude option? It allows you to directly ignore a certain pattern:

> fd --exclude '*.o' --exclude '/etc/secret-files' ...

Would this work for you?

Also, if I'm searching from ~, I don't expect the gitignore file several levels down in some project to get picked up by default.

If you don't expect them to be picked up, you can use the --no-ignore option.

alok · 2017-11-01T22:12:18Z

I think I may not have explained this clearly.

Say I have an ignore file at ~/.ignore and another one at ~/(several levels down)/.gitignore. My working directory is ~.

If I run fd --hidden --type directory ., I expect the files in ~/.ignore to be ignored, but not the ones in ~/(several levels down)/.gitignore. But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least).

--no-ignore would work if I could then specify a specific ignore file, hence the request for the new flag ``--ignore-file`.

tmccombs · 2017-11-02T04:49:55Z

But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least)

This is actually the behaviour I expect. If I am in a git repository I would expect fd to behave similar to git ls-files | grep. Usually, if I don't want to include files that are ignored by .gitignore files in their repository.

As @sharkdp mentioned the --include-file option in rg doesn't behave the way you describe. It adds ignore rules in addition to the ignore rules from .ignore and .gitignore files.

alok · 2017-11-02T05:29:13Z

I'm not in a git repo though. Hmm, I'll test something when I get in to work tomorrow

On Wed, Nov 1, 2017 at 9:49 PM Thayne McCombs ***@***.***> wrote: But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least) This is actually the behaviour I expect. If I am in a git repository I would expect fd to behave similar to git ls-files | grep. Usually, if I don't want to include files that are ignored by .gitignore files in their repository. As @sharkdp <https://github.com/sharkdp> mentioned the --include-file option in rg doesn't behave the way you describe. It adds ignore rules *in addition to* the ignore rules from .ignore and .gitignore files. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#156 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH8KTOCyxGHGFCsOikyXt-MiLlMwdc9nks5syUn0gaJpZM4QMFAH> .

-- - Alok

alok · 2017-11-02T23:11:00Z

My bad, its behavior is identical to rg. I still think the option to explicitly pass a set of ignore files and to only use those files for ignore patterns isn't a bad one, but I understand that's something extra.

rlue · 2017-11-10T11:27:23Z

I'd like to chime in in support of a command line option like this. ag and rg both honor both .gitignore and a separate .ignore file.

I'm currently using fzf to quick-find files in vim, supported by rg --files as my DEFAULT_FZF_COMMAND. Naturally, I have ripgrep set up to ignore all kinds of binary files (like images) that I would never want to open in vim, but which don't belong in .gitignore.

While I certainly wouldn't want fd to honor my .ignore file all the time (or even by default), it would be nice to have the option to.

sharkdp · 2017-11-10T20:28:53Z

I'd like to chime in in support of a command line option like this. ag and rg both honor both .gitignore and a separate .ignore file.

fd does that, too.

While I certainly wouldn't want fd to honor my .ignore file all the time (or even by default), it would be nice to have the option to.

Okay, so you would really like to have the --ignore-file flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.

rlue · 2017-11-11T00:39:34Z

fd does that, too.

So it does! My mistake. I think the bigger issue then is simply that it's not mentioned in the man page or the README.

Okay, so you would really like to have the --ignore-file flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.

Thanks; it'd definitely be nice to be able to rely on predefined ignore files to specify large sets of non-default ignore patterns.

rlue · 2017-11-11T07:12:08Z

After some more experimentation, it appears that fd also honors .rgignore by default (but not .agignore or .fdignore).

IMHO it would be saner to honor .gitignore and .ignore only, and leave .<xx>ignore files to apply to their respective utilities. If I want my rgignore rules to apply to fd, I'll either put them in .ignore or hardlink .rgignore and .fdignore to each other.

On a separate note, the manpage also contains an error (it says to use --type s for symlinks, where the actual option is --type l).

(If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)

See #156

sharkdp · 2017-11-11T08:46:05Z

After some more experimentation, it appears that fd also honors .rgignore by default (but not .agignore or .fdignore).

This is due to our usage of the ignore crate, that is also used by ripgrep. This crate has both .ignore and .rgignore hardcoded:

https://github.com/BurntSushi/ripgrep/blob/c4e194538472de2cd74664a9a016b9c25c0e800b/ignore/src/dir.rs#L213

However, if my research is correct, the .rgignore files (as well as the .agignore files for ag) are deprecated and were superseded by .ignore. So .rgignore only exists for legacy reasons and should not be used anymore.

However, you have a fair point, since users might very well want to ignore different files for fd as compared to rg/ag. If .ignore is really meant for search-in-file tools, we should probably not respect these files and introduce .fdignore files instead?

On a separate note, the manpage also contains an error (it says to use --type s for symlinks, where the actual option is --type l).

Thanks, fixed!

(If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)

That would be great! Any contributions are always very much appreciated.

rlue · 2017-11-11T14:56:42Z

However, if my research is correct, the .rgignore files (as well as the .agignore files for ag) are deprecated and were superseded by .ignore.

Yep, this is definitely true. From @BurntSushi himself:

Please use .ignore. .rgignore is deprecated.

If .ignore is really meant for search-in-file tools...

I can't say to a certainty what .ignore is for. AFAIK, it's a new convention that has been established by ag and rg. In my opinion, it's not a very good one — there are many programs that can be configured with ignorefiles: not just rg, ag, and fd, but git, syncthing, stow, and others. Why should a couple of popular grep utilities get an exemption on namespacing here?

As to how fd should handle this problem, I think the question should really be, how much deviation from find can you justify in the name of sane defaults? IMO, ignoring hidden files and git-ignored files is reasonable, but lumping in rg and ag ignores is pushing it. And I think the remainder of use cases could be reasonably resolved by supporting ~/.fdignore (ideally with an XDG-compatible alternative, like ~/.config/fdignore), which, again, could be hardlinked to ignore if users really want them to be the same.

But you're the boss here, so what do you think?

BurntSushi · 2017-11-11T15:19:24Z

I'd just like to chime in and say that I'd support simple changes to the ignore crate to iron out this UX, if y'all so choose.

sharkdp · 2017-11-12T14:54:52Z

IMO, ignoring hidden files and git-ignored files is reasonable, but lumping in rg and ag ignores is pushing it. And I think the remainder of use cases could be reasonably resolved by supporting ~/.fdignore (ideally with an XDG-compatible alternative, like ~/.config/fdignore), which, again, could be hardlinked to ignore if users really want them to be the same.

Agreed. Let's change this. So there are a few action points here:

Disable support for .ignore and .rgignore. I believe we simply have to remove the .ignore(..) call on the WalkBuilder. We should also carefully review the other options in WalkBuilder.
Add support for .fdignore. I think we should copy the behavior of ripgrep here (just with .fdignore instead of .ignore).
Add an --ignore-file option for users that would like to add a specific ignore file (again, in analogy to ripgrep).
Add integrations tests for this!
Document the new behavior (man page and README)

ptzz · 2017-11-19T19:01:21Z

Currently the fd man page says

-I, --no-ignore
              Do not respect any gitignore(5) files.

The description is wrong since it also ignores the .rgignore file.

It would be useful to have an option --no-ignore-vcs specifically targeting git/vcs ignore files (same as ripgrep has).

ptzz · 2017-11-26T21:40:17Z

Disable support for .ignore and .rgignore. I believe we simply have to remove the .ignore(..) call on the WalkBuilder. We should also carefully review the other options in WalkBuilder.
Add support for .fdignore. I think we should copy the behavior of ripgrep here (just with .fdignore instead of .ignore).

I gave this a try but I think it's hard to get right without modifying the hardcoded .ignore and .rgignorenames in the ignore crate.

My attempt was to add .fdignore as an explicit ignore using add_ignore(), and disabling .ignore/.rgignore using ignore(false). That 1) causes precedence issues (.fdignore lower than .gitignore), and 2) add_ignore() only adds a global ignore file, whereas .ignore is respected in any traversed directory.

I think making the names of the default ignore files (currently static IG_NAMES) configurable in the ignore crate API is the best option. @BurntSushi does this sound like a reasonable change?

BurntSushi · 2017-11-28T00:15:59Z

@ptzz Please see the latter half of this comment: BurntSushi/ripgrep#673 (comment)

TL;DR - While I believe that making IG_NAMES configurable is the shortest path to your destination, I would actually like to keep .ignore as a standard name used by the ignore crate. If we stick to that, then I think the way to solve your problem is to add an additional layer of ignore files that has higher precedence than .ignore but uses a custom (probably application specific) name. For example, ripgrep might use .rgignore where as fd might use .fdignore, but both tools could still respect .ignore if ignore(true) is set.

ptzz · 2017-11-28T07:00:12Z

@BurntSushi That sounds like a great approach!

Btw., when trying the workarounds above I failed to find a way to respect only the additional ignore file. Using

ignore(false)
git_ignore(false)
add_ignore(<fdignore_abspath>)

Did not work. But with git_ignore(true), fdignore was also respected. I did however not investigate why due the other reasons this approach would not work. Maybe it’s expected behavior?

BurntSushi · 2017-11-28T12:37:38Z

If that's true that sounds like a bug to me.

…

On Nov 28, 2017 2:00 AM, "ptzz" ***@***.***> wrote: @BurntSushi <https://github.com/burntsushi> That sounds like a great approach! Btw., when trying the workarounds above I failed to find a way to respect *only* the additional ignore file. Using ignore(false) git_ignore(false) add_ignore(<fdignore_abspath>) Did not work. But with git_ignore(true), fdignore was also respected. I did however not investigate why due the other reasons this approach would not work. Maybe it’s expected behavior? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#156 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAb34tHZqFBwcZyGWkzMpQG019DyQ43eks5s669-gaJpZM4QMFAH> .

This commit adds support for ignore files with custom names. This allows for application specific ignorefile names, e.g. using `.fdignore` for `fd`. See also: #673 See also: sharkdp/fd#156

BurntSushi · 2018-01-29T21:06:33Z

Thanks to @ptzz's hard work, BurntSushi/ripgrep#706 got merged which will be in the next release of ignore!

sharkdp · 2018-02-13T22:18:09Z

ignore-0.4.0 has been released (thanks!), but it's currently not possible to implement this due to BurntSushi/ripgrep#800

sharkdp · 2018-02-21T21:59:32Z

Added support for .fdignore via #241.

sharkdp · 2018-03-13T21:59:20Z

We now have:

--exclude/-E to exclude specific glob patterns.
Support for .fdignore files.
--no-ignore-vcs to show .gitignored files (but not the .fdignored ones).

All this has been documented and we have integration tests.

I currently don't see any need for the additional --ignore-file flag, but if someone has a strong argument for it, I am happy to discuss this again (and reopen the ticket). Let me know what you think!

ptzz · 2018-03-20T04:08:23Z

I'm currently using fzf to quick-find files in vim, supported by rg --files as my DEFAULT_FZF_COMMAND. Naturally, I have ripgrep set up to ignore all kinds of binary files (like images) that I would never want to open in vim, but which don't belong in .gitignore.

@sharkdp I have a similar setup. With support for --ignore-file, fd could be used instead of rg.

I.e. by default I want fd to find any file type, but it would be useful with a simple way to have it return only textual files.

sharkdp · 2018-03-20T20:44:45Z

@ptzz I see. That's a valid use case... a .fdignore file would always be present.

I'm reopening this ticket to track the addition of --ignore-file.

sharkdp · 2018-03-26T06:28:49Z

Implemented in #271

sharkdp · 2018-03-26T09:52:15Z

Both .fdignore support as well as --ignore-file are part of fd v7.0.0

mbrubeck · 2018-10-08T20:06:14Z

Just as feedback, I miss having a shared .ignore file that works out of the box with both fd and rg. Now I have to create duplicate files for all my projects, or remember to pass extra arguments every time I call fd. (And when finding across multiple directory trees at once, there's no single argument that will pull in all the .ignore files, which used to happen automatically.)

Maybe someday there could be a configuration file that adds the old behavior as an option...

BurntSushi · 2018-10-08T20:09:56Z

@mbrubeck Oh, does fd not support .ignore? I might be misremembering, but I thought the plan was for both ripgrep and fd to support .ignore, while also supporting application specific files with higher precedence (e.g., .rgignore and .fdignore, respectively).

sharkdp · 2018-10-08T20:14:41Z

Oh, does fd not support .ignore?

It does not, anymore. We removed it (and added .fdignore) because some people argued that they would typically put more ignore-patterns into the .ignore file (which is intended for ripgrep-like tools, if I understand correctly) as compared to the .fdignore file. For example, you would maybe .ignore PDF files because you do not want to search through them, but you would still like to be able to find those files.

If I got this wrong or if there is a strong support for adding .ignore support back into fd, I'm happy to discuss this again.

BurntSushi · 2018-10-08T20:19:32Z

For example, you would maybe .ignore PDF files because you do not want to search through them, but you would still like to be able to find those files.

Right. You could achieve this by putting *.pdf in your .rgignore but not your .fdignore. Alternatively, if fd supported .ignore and .fdignore and *.pdf was in your .ignore, then you could put !*.pdf in your .fdignore, which would override the *.pdf in your .ignore.

I think I thought the idea was that .ignore could be shared across applications, but that the ignore crate now also permits application specific ignore files so that users could share as many rules as they like while also being able to override rules on a per-app basis. It gives the most flexibility, anyway.

The downside I can see is that if someone did treat .ignore as ripgrep specific rules, then they might get frustrated if fd also respected that if they didn't know how to override it (or if they didn't know about .rgignore).

sharkdp · 2018-10-08T20:32:19Z

I think I thought the idea was that .ignore could be shared across applications, but that the ignore crate now also permits application specific ignore files so that users could share as many rules as they like while also being able to override rules on a per-app basis. It gives the most flexibility, anyway.

If this is the idea, I'm all for it! My previous understanding was that .ignore files were only meant to be used by the "search-in-file tools". Personally, I also share your and @mbrubecks opinion and would like to be able to just put my rules into a .ignore file (if it doesn't fit into .gitignore, of course).

The downside I can see is that if someone did treat .ignore as ripgrep specific rules, then they might get frustrated if fd also respected that if they didn't know how to override it (or if they didn't know about .rgignore).

I'm okay with this. We can add a detailed explanation of how .ignore and .rgignore/.fdignore are meant to be used in fds README.

ptzz · 2018-10-08T20:32:41Z

I also thought .ignore would be shared across applications. My current workaround is to have .fdignore contain what should by ignored by both rg and fd, and alias rg to rg --ignore-file .fdignore. Currently I don't use .ignore at all, only .rgignore and .fdignore.

If .ignore would have been shared, my .fdignore would have been called .ignore and there would be no need for the rg alias.

sharkdp · 2018-10-08T20:37:29Z

Let's add back support for .ignore then 👍

sharkdp · 2018-10-27T16:48:22Z

Support for .ignore has been re-enabled in fd-7.2.0.

sharkdp added feature-request idea labels Nov 1, 2017

alok closed this as completed Nov 2, 2017

sharkdp reopened this Nov 10, 2017

sharkdp added the good first issue label Nov 10, 2017

sharkdp added a commit that referenced this issue Nov 11, 2017

Update man page entry for '--type'

aea8fe5

See #156

rlue mentioned this issue Nov 11, 2017

Why was .rgignore deprecated in favor of .ignore? BurntSushi/ripgrep#673

Closed

ptzz mentioned this issue Nov 21, 2017

Add --no-ignore-vcs option #175

Merged

ptzz mentioned this issue Dec 4, 2017

Support ignore files with custom names BurntSushi/ripgrep#706

Merged

sharkdp changed the title ~~request: add --ignore-file flag~~ Add --ignore-file flag Feb 10, 2018

sharkdp removed good first issue idea labels Feb 19, 2018

sharkdp mentioned this issue Feb 21, 2018

Add support for .fdignore files #241

Merged

sharkdp added a commit that referenced this issue Mar 13, 2018

Documentation for -E and .fdignore, see #156

87d0ccd

sharkdp closed this as completed Mar 13, 2018

sharkdp added this to the 7.0 milestone Mar 13, 2018

sharkdp reopened this Mar 20, 2018

sharkdp mentioned this issue Mar 25, 2018

Implement '--ignore-file' #271

Merged

sharkdp closed this as completed in #271 Mar 26, 2018

sharkdp reopened this Oct 8, 2018

sharkdp mentioned this issue Oct 8, 2018

Loosen strict handling of missing --ignore-file? #280

Closed

sharkdp modified the milestones: 7.0, v7.2.0 Oct 27, 2018

sharkdp closed this as completed in 6ff58ab Oct 27, 2018

sharkdp added a commit that referenced this issue Oct 27, 2018

Properly use .ignore files, see #156

f064ee5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `--ignore-file` flag #156

Add `--ignore-file` flag #156

alok commented Oct 31, 2017

sharkdp commented Nov 1, 2017

alok commented Nov 1, 2017 •

edited

Loading

tmccombs commented Nov 2, 2017

alok commented Nov 2, 2017 via email

alok commented Nov 2, 2017

rlue commented Nov 10, 2017

sharkdp commented Nov 10, 2017

rlue commented Nov 11, 2017

rlue commented Nov 11, 2017 •

edited

Loading

sharkdp commented Nov 11, 2017

rlue commented Nov 11, 2017

BurntSushi commented Nov 11, 2017

sharkdp commented Nov 12, 2017 •

edited

Loading

ptzz commented Nov 19, 2017 •

edited

Loading

ptzz commented Nov 26, 2017

BurntSushi commented Nov 28, 2017 •

edited

Loading

ptzz commented Nov 28, 2017

BurntSushi commented Nov 28, 2017 via email

BurntSushi commented Jan 29, 2018

sharkdp commented Feb 13, 2018

sharkdp commented Feb 21, 2018

sharkdp commented Mar 13, 2018

ptzz commented Mar 20, 2018

sharkdp commented Mar 20, 2018 •

edited

Loading

sharkdp commented Mar 26, 2018

sharkdp commented Mar 26, 2018

mbrubeck commented Oct 8, 2018

BurntSushi commented Oct 8, 2018

sharkdp commented Oct 8, 2018

BurntSushi commented Oct 8, 2018

sharkdp commented Oct 8, 2018

ptzz commented Oct 8, 2018

sharkdp commented Oct 8, 2018

sharkdp commented Oct 27, 2018

Add --ignore-file flag #156

Add --ignore-file flag #156

Comments

alok commented Oct 31, 2017

sharkdp commented Nov 1, 2017

alok commented Nov 1, 2017 • edited Loading

tmccombs commented Nov 2, 2017

alok commented Nov 2, 2017 via email

alok commented Nov 2, 2017

rlue commented Nov 10, 2017

sharkdp commented Nov 10, 2017

rlue commented Nov 11, 2017

rlue commented Nov 11, 2017 • edited Loading

sharkdp commented Nov 11, 2017

rlue commented Nov 11, 2017

BurntSushi commented Nov 11, 2017

sharkdp commented Nov 12, 2017 • edited Loading

ptzz commented Nov 19, 2017 • edited Loading

ptzz commented Nov 26, 2017

BurntSushi commented Nov 28, 2017 • edited Loading

ptzz commented Nov 28, 2017

BurntSushi commented Nov 28, 2017 via email

BurntSushi commented Jan 29, 2018

sharkdp commented Feb 13, 2018

sharkdp commented Feb 21, 2018

sharkdp commented Mar 13, 2018

ptzz commented Mar 20, 2018

sharkdp commented Mar 20, 2018 • edited Loading

sharkdp commented Mar 26, 2018

sharkdp commented Mar 26, 2018

mbrubeck commented Oct 8, 2018

BurntSushi commented Oct 8, 2018

sharkdp commented Oct 8, 2018

BurntSushi commented Oct 8, 2018

sharkdp commented Oct 8, 2018

ptzz commented Oct 8, 2018

sharkdp commented Oct 8, 2018

sharkdp commented Oct 27, 2018

Add `--ignore-file` flag #156

Add `--ignore-file` flag #156

alok commented Nov 1, 2017 •

edited

Loading

rlue commented Nov 11, 2017 •

edited

Loading

sharkdp commented Nov 12, 2017 •

edited

Loading

ptzz commented Nov 19, 2017 •

edited

Loading

BurntSushi commented Nov 28, 2017 •

edited

Loading

sharkdp commented Mar 20, 2018 •

edited

Loading