Support persistent configuration (`.rgrc`?)

[SimenB]

I'd love to keep different types for instance, and not have to specify --type-add each time.

My specific case is type zsh which searches zsh config files (.zshrc *.zsh, and some custom .zsh*).

Maybe it could support more stuff like no-heading, --heading etc as well, which would allow people to customize rg more.

(I know I can just alias rgzsh="rg --type-add 'zsh:*.zsh' --type-add 'zsh:.zshrc' --type zsh", but meh)

Loving the tool, btw!

[BurntSushi]

Can you make do with creating an alias? Or even a script in your ~/bin? For example, I have ~/bin/rgp:

#!/bin/sh

exec rg -p "$@" | less -RFX

[SimenB]

I just updated the OP with alias stuff 😄

And yes, I could, but then I have to remember my aliases (woe is me, right?). Doing rg -tzsh is more natural when I've gotten used to types than doing rgzsh or whatever.

rgp makes sense, though!

[BurntSushi]

Why don't you just alias rg itself? This works, for example:

alias rg="rg --type-add 'zsh:*.zsh' --type-add 'zsh:.zshrc'"

Then you can do rg -tzsh.

(Also, zsh seems like a common enough type that it should be part of ripgrep proper! :-))

[SimenB]

Aliasing rg seems pretty good!

I can provide a PR for zsh type (just add an entry here I suppose: https://github.com/BurntSushi/ripgrep/blob/master/src/types.rs)

But I'd still really like for that to be defined for rg, and not drowned amongst all of my other aliases (I still do alias | rg whatever since I forget. I'd like to think rg --type-list is nicer (and yes, I realize rg --type-list would work with your suggested rg alias)).

[BurntSushi]

I can provide a PR for zsh type (just add an entry here I suppose: https://github.com/BurntSushi/ripgrep/blob/master/src/types.rs)

Sounds good!

But I'd still really like for that to be defined for rg, and not drowned amongst all of my other aliases (I still do alias | rg whatever since I forget. I'd like to think rg --type-list is nicer (and yes, I realize rg --type-list would work with your suggested rg alias)).

You can also do command -V rg to see the alias.

[SimenB]

Conclusion: I can achieve what I want using simple aliases. I'd still like an .rgrc file though, to match .ackrc, .ptconfig.toml etc. 😄 I do understand if you feel like aliasing is enough, though

[SimenB]

Or maybe support $RG_OPTIONS to match grep? It just seems more explicit than aliasing rg itself.

[BurntSushi]

I do feel like aliases are a pretty good solution to this problem, and I'd like to stick with them for now.

I think we should leave this ticket open so that others can chime in. For example, aliases may not be easy to define on all platforms.

[SimenB]

BTW, is it possible to not recurse through directories? Case in point is searching for something in a zsh file in$HOME. Having rg recurse through everything is slow as hell (naturally).

Additionally: Is it possible to add ignore patterns? I managed to hit chrome application cache, which is an absolutely monstrous file

[BurntSushi]

BTW, is it possible to not recurse through directories?

Yes:

    --maxdepth NUM
        Descend at most NUM directories below the command line arguments.
        A value of zero only searches the starting-points themselves.

Additionally: Is it possible to add ignore patterns? I managed to hit chrome application cache, which is an absolutely monstrous file

Yes. Create a .ignore file and add patterns to that. (Or add them to your .gitignore.)

[SimenB]

Ok, but not add ignore to the execution itself?

Will an .rgignore file be respected? I just feel having an .ignore file in $HOME might be confusing if it's just used for rg

[BurntSushi]

Ok, but not add ignore to the execution itself?

I don't understand the question.

Will an .rgignore file be respected?

Please use .ignore. .rgignore is deprecated. Where did you see .rgignore?

I just feel having an .ignore file in $HOME might be confusing if it's just used for rg

I don't understand what's confusing about it. The Silver Searcher supports the same format.

The .ignore file can be anywhere. It could live right next to the thing you want to ignore. It works just like gitignore.

[BurntSushi]

Perhaps you're looking for the -g/--glob flag?

[SimenB]

I don't understand the question.

I want to do e.g. rg --ignore-dir 'Application Cache' or something like it, not have to create a file on disk.

Where did you see .rgignore?

Nowhere, it was just a name proposal. I can guess you don't like it though 😁

Perhaps you're looking for the -g/--glob flag?

That should work! Just rg -g '!Application Cache/'?

BTW, using --maxdepth 0 never works, I always get

No files were searched, which means ripgrep probably applied a filter you didn't expect. Try running again with --debug

Doing --maxdepth 1 seems to just search current dir, though. Bug or should the man file get an update?

[BurntSushi]

That should work! Just rg -g '!Application Cache/'?

You probably want rg -g '!**/Application Cache/**', since the glob is applied to the full file path.

Doing --maxdepth 1 seems to just search current dir, though. Bug or should the man file get an update?

Seems correct to me. Read the docs again please:

    --maxdepth NUM
        Descend at most NUM directories below the command line arguments.
        A value of zero only searches the starting-points themselves.

A value of zero only searches the starting points. So if you run rg --maxdepth 0 foo, then the starting point is the current directory, which obviously can't be searched, since it isn't file. If you do rg --maxdepth 1 foo, then it descends one level. These are the same semantics as find, e.g., see the output of find ./ -maxdepth 0 and find ./ -maxdepth 1.

[SimenB]

Ah of course, so --maxdepth 0 is only when searching a file? Makes sense when I read it again!

Thanks for answering all of my questions, super helpful! 😄 Especially the -g one will be useful

[BurntSushi]

Ah of course, so --maxdepth 0 is only when searching a file?

--maxdepth can be used any time. All it does is control the number of times it descends into a directory. If the value is zero, then it will never descend into any directories. Therefore, since rg foo is equivalent to rg foo ./, the directory ./ isn't descended into, so there's nothing to search.

It's most likely that a value of 0 is only useful in a generic context, e.g., when you don't know if your file parameters will be files, directories or a mixture and you don't want to do any directory recursion.

[SimenB]

A more useful thing for --maxdepth doc might be "A value of one will only search files in the passed in directory" or something like it. But now I know, so I'm good 😄

[kastiglione]

I've hit one issue with using an alias: zsh completion doesn't work when I've shadowed the rg command with a rg alias. When I unalias rg the completion works. There's probably some way to get the zsh completion system to handle this case, but I don't yet know how to do that.

EDIT: The answer is setopt complete_aliases.

[gvol]

Just my two cents, but I really like .ackrc because it's used at the command line and when invoked by other tools like Emacs (a place that aliases don't work).

[BurntSushi]

In that case, can you put a wrapper script in your $HOME/bin?

On Nov 23, 2016 00:08, "Ivan Andrus" notifications@github.com wrote:

Just my two cents, but I really like .ackrc because it's used at the
command line and when invoked by other tools like Emacs (a place that
aliases don't work).

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#196 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAb34niAsM6kJe2XlAWQGP9sFz0-gg5dks5rA8pWgaJpZM4Kieh1
.

[BurntSushi]

I know of at least one Windows user who wants this pretty badly. (cc @retep998) It sounds like the claim is that, on Windows at least, defining aliases and/or wrapper scripts is neither common nor convenient to do. Is this others' experience as well? Could someone please outline exactly what one has to do on Windows to create a wrapper script? Is it really an unreasonable expectation?

If we go down this path, we should consider the following:

• Whether clap can give this to us for free some day: clap-rs/clap#748
• If clap doesn't, then we need to come up with our own specification.
• For example, where are config files read from? On *nix, I'm familiar with the xdg basedir specification, so we could follow that, but I don't know what the idioms are on Windows.
• We may need to solve problems like "I set --context 5 in my config file, but I want to override occasionally with --context 0 on the command line." We can't just trivially merge the config with the CLI parameters since rg --context 5 --context 0 foo is illegal today. This is probably true of several other flags, and many other flags probably need the opposites added. e.g., If --no-ignore is in the config file, how does a user re-enable ignore functionality?

IMO, the above implies significant complexity and likely an easy source of bugs. If we moved forward with this, I think I'd have to demand a rock solid specification (which I'm not inclined to work on any time soon) or some kind of conscious concessions at least (like, "not every option is overridable," for example, which seems like bad UX honestly).

[BurntSushi]

OK... So I've hit another snag. As far as I can tell, nobody has yet solved this problem of configuration files in Rust-land. What I'd be after is something that lets me read configuration files in a way that is idiomatic on Linux, macOS and Windows. There are various (many, actually) crates that purport to do something in this space, but I don't feel comfortable using any of them for various reasons (mostly, they are unmaintained). I personally do not want to jump into those waters as I've observed how dicey they can be from afar.

Therefore, I suspect the initial implementation of this feature is going to land behind a flag. That is, rg --config /wherever/you/want/.ripgreprc. We'll definitely want a flag like that anyway. This is probably fine on Unix-like systems where creating an alias is easy, but I guess doing something similar on Windows is not as easy. Which is kind of a bummer since Windows is kind of what pushed me over the edge to accept this feature request.

We could start with "project specific" config files. That is, ripgrep will look in the CWD and its parents for a .ripgreprc file and load it automatically. One possible thing that somewhat worries me here is that it will encourage folks to add a $HOME/.ripgreprc file, which will work in many cases, but it won't actually be a global config. So when you cd into /etc (or whatever) you might be surprised to discover that your config isn't being applied.

Thoughts?

[BurntSushi]

Another thought I had was permitting arguments to be specified in an environment variable. Both ack and grep support it via ACK_OPTIONS and GREP_OPTIONS, although I've heard Ack's author say that they would like to remove support for it, and GNU grep's man page says this:

   GREP_OPTIONS
         This  variable  specifies default options to be placed in front of any explicit options.  As this causes problems when writing portable scripts, this feature will be removed in a future release
         of grep, and grep warns if it is used.  Please use an alias or script instead.

But the particular downside mentioned by grep here also strikes me as a downside of having auto-loaded config files too.

There is also the question of precedence for a hypothetical RIPGREP_OPTIONS environment variable. Is it loaded before all configs? After all configs? Before configs specified on the CLI but after any auto-loaded configs?

Questions abound...

[kbknapp]

These were all the questions that started to put clap-rs/clap#748 as well 😟

The conclusion I've come to (and just not implemented it yet) is allowing "additional argv"s to be declared and parsed prior to the CLI (because of overrides, the CLI [IMO] should always be last and therefore able to override everything else).

I don't really want clap to have to worry about loading config files in tons of different formats, file I/O, transforming a config into an argv, etc...so I'd prefer all that happen in an external crate or in application code.

Once all that has happened, and we have something that looks like an argv, I'd like to make it as simple as saying: app.argv("some --args=to -be --parsed -b=efore the --cli").get_matches(). Of course, an ENV var may be the only exception to this, as clap already read ENV vars for argument values, so adding something like app.env_argv("RIPGREP_OPTIONS").argv("options --from-some -config=file").get_matches() wouldn't be out of the question.

My personal thoughts are the evaluation order should be:

• ENV vars
• Global config files
• Project config files
• CLI

Where each step can override any values from the step above it.

Edit: Note the "config file" ones are the only ones were clap couldn't control the order itself, and it would be up to application code to specify the order (i.e. if they'd prefer global configs be able to override project configs).

[kbknapp]

Oh, that issue also brings up some more questions about, imagine an argument which accepts multiple values. If one value is defined in the ENV, one in a config file, and one at the CLI what should the matches look like? All three values? Only the CLI?

[okdana]

So it might be the case here that we actually want to fix the glob parser to interpret some escape sequences, but I suppose that needs some careful thought and inspection as to whether other glob engines do the same.

As far as i'm aware, there aren't (m)any that do. fnmatch(3) doesn't have any special handling for it at least: \t is treated the same as t. git doesn't seem to do anything special with it either. Anyway, related to #526 probably.

It seems like we could start with the ackrc format basically as-is, and then build conveniences as we go. We might also want to consider a strategy for evolving the format in a non-breaking way, but that is profligate with its own set of unique challenges and trade offs...

Yeah, the only concern with starting out simple is how much it could eventually break. e.g., if you add quoting or escape sequences or whatever, who knows what'll happen to people's existing files. But it is still 0.x software after all, so it's not that big a deal i suppose.

We could start with "project specific" config files.

I'm not familiar with any other tools that do this but this would bother me and i would immediately be looking for a way to turn it off. Project-specific ignore rules, OK, i get that, but i don't want random people screwing with my actual rg options.

Thoughts?

I don't think i understand the issue exactly. It sounds like you're more worried about finding the config file than parsing/loading it? Is there a reason you wouldn't just check for $HOME/.ripgreprc and $XDG_CONFIG_HOME/ripgrep/ripgreprc (or whatever the Linux standard is)?

But the particular downside mentioned by grep here also strikes me as a downside of having auto-loaded config files too.

AFAIK the main reason GNU are removing GREP_OPTIONS is that grep is used in loads of shell scripts (some of them quite critical), and people would constantly forget to unset GREP_OPTIONS at the top of those scripts, and it'd cause all kinds of bizarre (maybe even security-impacting) behaviour for them.

Since rg is probably not going to be used in any super important shell scripts, i don't think it's a big concern for this project. The only issue with supporting an environment variable is that you (or clap) have to have a mechanism for parsing options from a 'flat' string.

There is also the question of precedence for a hypothetical RIPGREP_OPTIONS environment variable.

I would expect an environment variable to override any config-file options, based on the fact that the former is ephemeral and easier to modify than the latter. That's how git works too: config-file options -> environment-variable options -> command-line options

[BurntSushi]

@kbknapp TL;DR I think any specific issues with clap for config in ripgrep are actually resolved on my end. See the latest few commits in my ag/persistent-config branch. :-)

Once all that has happened, and we have something that looks like an argv, I'd like to make it as simple as saying: app.argv("some --args=to -be --parsed -b=efore the --cli").get_matches().

I think I can just do this already? clap::App has a get_matches_from, and I can just build the argv myself by concatenating all of the flags from the config files, and making sure env::args comes last. Or maybe you're just talking about convenience routines here?

[EDIT] - Ah I see, you're talking about one big string that clap then parses. Yeah that'd be interesting.

Oh, that issue also brings up some more questions about, imagine an argument which accepts multiple values. If one value is defined in the ENV, one in a config file, and one at the CLI what should the matches look like? All three values? Only the CLI?

I've got this worked out now. :-) See 90387bb and in particular the RGArgKind type which has some docs on it.

I should revise what I just said. I have it worked out for ripgrep. I suspect solving this in a general case is far harder and there are probably lots of variants. I don't envy your task! To be honest though, I'm actually pretty happy with my work-around thus far. It caused me to clean up my arg definitions, and the changes to the code that interfaces with clap::ArgMatches were actually very small. Basically, it was just a matter of changing the value_of methods to return the last value instead of the first value.

@okdana

I'm not familiar with any other tools that do this but this would bother me and i would immediately be looking for a way to turn it off. Project-specific ignore rules, OK, i get that, but i don't want random people screwing with my actual rg options.

Interesting! ack has this feature AFAIK. Best to start without it then. :-)

I don't think i understand the issue exactly. It sounds like you're more worried about finding the config file than parsing/loading it? Is there a reason you wouldn't just check for $HOME/.ripgreprc and $XDG_CONFIG_HOME/ripgrep/ripgreprc (or whatever the Linux standard is)?

Ah yeah I should have said more words. The XDG_CONFIG_HOME style works fine on Linux (and maybe Mac?), but from what I understand it isn't a good default on Windows. So while I'd probably be OK writing out the XDG logic for Linux (I've done it before, it's not that hard), I'm far less in tune with how other platforms do this, so I'm very hesitant to open that can of worms.

The only issue with supporting an environment variable is that you (or clap) have to have a mechanism for parsing options from a 'flat' string.

Yup, good point. Drats. That kind of deflates that idea then!

I would expect an environment variable to override any config-file options, based on the fact that the former is ephemeral and easier to modify than the latter. That's how git works too: config-file options -> environment-variable options -> command-line options

Yeah I think that is consistent with Ack but actually inconsistent with @kbknapp's thoughts above! Regardless, it seems wise to punt on the auto-loading aspect of this ticket for now anyway. If project specific configs are contentious (and you make a good point), then the other variant of autoloading is to look in platform specific directories where application configs are normally stored, and as far as I can tell, the Rust ecosystem doesn't offer anything in that space that I'd be comfortable using just yet.

[BurntSushi]

@nagisa on IRC suggested an env var that could be used to specify the location of a config file. It's slightly roundabout, but I think would be a nice sweet spot. It would sidestep the Windows issue of needing to write an alias/wrapper script, and also sidestep the issue of needing to parse an argv out of an env var itself.

[BurntSushi]

(Hah, if we support a --config-file flag, then it will require ripgrep to recursively load them... sigh)

Maybe only the env var for now? ^_^

[kbknapp]

Yeah I think that is consistent with Ack but actually inconsistent with @kbknapp's thoughts above!

To be clear, I'm not stuck in a position either way, those were just my initial thoughts. The only thing I have strong fieelings about is CLI should override all, which it sounds like is a universa thought 😉

[BurntSushi]

which it sounds like is a universa thought

Yes indeed!

[BatmanAoD]

Some thoughts/comments:

• I would be interested in supporting and/or maintaining a library for generically finding config files and sequencing (but not parsing) them appropriately. But I don't have a lot of time. @BurntSushi, were any of the libraries you looked at that aren't maintained look like promising starts?
• Windows does have a concept of a $HOME directory, %USERPROFILE%. For XDG configuration, I believe the standard (e.g. used by nvim) is %APPDATA%. I think it would be sufficient to look in those two directories for config files.
• An environment variable containing a single config file path (or more, if they could be split appropriately) would be a perfectly acceptable solution. I am highly in favor of this.
  • This could actually be quite useful for scripting, because the script could check some aspect of the environment and select a config file as appropriate. See for instance this customization I use on Windows but not elsewhere.
• I don't think a --config-file flag would be necessary, but something like Vim's -u NONE would, I think, be very important to support scripting. This option would prevent ripgrep from using any custom configuration, even if a config file is specified in an environment variable. I suggest --no-config or something similar.

[BurntSushi]

@BatmanAoD Sounds great! When I did my survey on available crates, I basically dismissed almost all of them a priori without even looking at the code based on one (or more) of the following reasons:

1. It was effectively unmaintained. (By this, I mean there was very little recent commit activity and an issue/PR backlog with seemingly important things that need to be addressed.)
2. The documentation was either missing or insufficient.
3. It wasn't under a permissive license.
4. Some combination of the above coupled with the fact that nobody was using it. If nobody is using a library, then that means I need to do a thorough audit before bringing it in as a dependency.
5. The crate addressed XDG but nothing else.

So on those grounds they basically all failed the sniff test before I was willing to spend more time with them. So in that sense, I don't know if any were a good start. You'll need to do your own review. :-)

If you do take this on, beware the can of worms. Here are some relevant links:

• rust-lang/cargo#2127 (yikes)
• rust-lang/rustup.rs#247
• rust-lang/cargo#1734
• rust-lang/cargo#148

[BurntSushi]

I suggest --no-config or something similar.

Definitely. This will be in the initial support for persistent config.

[wsdjeg]

If you want to add support for .rgrc, please consider the configuration file in HOME directory or the root of a project.

I mean if I have a .rgrc which path is ~/src/Foo/.rgrc, and then if I run rg command from the subdirectories， such as ~/src/Foo/src/utils/.

[BurntSushi]

@wsdjeg Please see comments above. I think we're going to punt on everything for now except for setting the config file path via an env var.

[wsdjeg]

oh, sorry, use evn var seems better. 👍

[BurntSushi]

PR submitted: #772

[jason-s]

I was going to ask what the eventual behavior was decided, but you wrote it up quite well in the commit message (a very helpful example of a good commit msg btw):

This commit adds support for reading configuration files that change
ripgrep's default behavior. The format of the configuration file is an
"rc" style and is very simple. It is defined by two rules:

  1. Every line is a shell argument, after trimming ASCII whitespace.
  2. Lines starting with '#' (optionally preceded by any amount of
     ASCII whitespace) are ignored.

ripgrep will look for a single configuration file if and only if the
RIPGREP_CONFIG_PATH environment variable is set and is non-empty.
ripgrep will parse shell arguments from this file on startup and will
behave as if the arguments in this file were prepended to any explicit
arguments given to ripgrep on the command line.

For example, if your ripgreprc file contained a single line:

    --smart-case

then the following command

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo

would behave identically to the following command

    rg --smart-case foo

This commit also adds a new flag, --no-config, that when present will
suppress any and all support for configuration. This includes any future
support for auto-loading configuration files from pre-determined paths
(which this commit does not add).

Conflicts between configuration files and explicit arguments are handled
exactly like conflicts in the same command line invocation. That is,
this command:

    RIPGREP_CONFIG_PATH=wherever/.ripgreprc rg foo --case-sensitive

is exactly equivalent to

    rg --smart-case foo --case-sensitive

in which case, the --case-sensitive flag would override the --smart-case
flag.

Closes #196

[elentok]

Awesome! thanks!

When are you planning to create a new release?

[BurntSushi]

@elentok Soon. Releases happen when they happen. I do this in my spare time, so I will never set an explicit schedule.

[kbknapp]

I was re-reading through old messages in this thread and noticed the # allow omission of leading --. Forgive me if this has already been addressed, but I wanted throw one last thought out there about this. I actually like that thought, but there is one edge case that is possible and could lead to strange errors.

Assuming these are equivalent:

-H

# or

H

Since clap allows combining short switches (i.e. -Ha is the same as -H -a), and even combining with a trailing option/flag (i.e. -Hatyaml is the same as -H -a -tyaml is the same as -H -a -t=yaml etc.). One would need to know if what was found in the config file sans -- or - was meant to be a long style flag --word or combination of short switches.

Since ripgrep has quite a few short switches, I'd says it's possible (not sure how likely) a combination of short switches would also equal a long flag sans --.

The fix for this would be to document, IF the user intends to combine switches in a ripgrep config, they should add a leading - otherwise it'd be treated as a long flag.

Just some thoughts 😄

[BurntSushi]

@kbknapp Interesting. Good thing I did not implement that shortcut. Sounds like a good reason to just not do it!

[okdana]

I think i had in mind (a) only one option per line and/or (b) not even supporting the short options, since (i believe) that's how the other tools i mentioned work. But yeah you couldn't omit the -- having mixed them

[danemacmillan]

I know I'm late to the discussion, especially since you've (@BurntSushi) already submitted a PR, but I do like how rclone handles persistent configs as environment variables: https://rclone.org/docs/#environment-variables.

[BurntSushi]

@danemacmillan I'm happy with the approach I took.

[BurntSushi]

@danemacmillan

already submitted a PR

Not only that, but the PR has been merged and is in the 0.8.0 release!