String match with capturing group

[m0nhawk]

fish seems to broke the behavior of string match.

I'm trying to match the part of filename after _ and before .:

➜ string match -r "_([^_]*)\." "very_nice_file_whatmatter.csv"
_whatmatter.
whatmatter

➜ string match -r "_[^_]*\." "very_nice_file_whatmatter.csv"
_whatmatter.

I found this issue #4925, but the behavior there is different than what I got.

Not sure if it's the bug or a feature I'm missing.

fish, version 3.0.2

[krobelus]

This looks like expected behavior to me. You can add a filter to extract the capture group:

string match -r "_([^_]*)\." "very_nice_file_whatmatter.csv" | awk 'NR == 2'

(or NR%2 == 0 if you have more inputs)

[m0nhawk]

And was the behavior changed? Because the code in the linked issue (comment) yielding different result for me:

➜ string match -r '/.*/' -- some/weird/stuff
/weird/

I know I can refine the output, but it looks like a regression or a bug, if it changed since that comment.

[faho]

Because the code in the linked issue (comment) yielding different result for me:

No, that was just an oversight. It was always "/weird/".

If you want to have just a certain part of the input, use string replace.

Also I'm preparing an additional "--groups-only" option to string match that will just print the capturing groups, not the full match as well.

[mqudsi]

The full match is considered a group, every single regex implementation that I know of defines the entire match to be capture group 0, and further explicit capturing groups are numbered from there.

[m0nhawk]

@mqudsi I was expecting it to be multiple outputs, but the issue confuses me and I wanted to clarify.

@faho The --groups-only argument would be awesome. Thanks! 👍

[mqudsi]

I just ran into this with 912421f, which could have been much shorter if there were some way to change the default behavior.

I don't think --groups-only goes far enough, and I think at the same time it is too rigid. I was giving this some thought in the shower, and I think I would prefer an extension to string match -r along the lines of string replace -r with an additional parameter describing what to output, e.g.

echo "foo bar baz" | string match -r 'foo (bar) baz' '$2' 

would only print the second group (the first explicitly captured group). To do this now requires using string replace -r and inverting the logic, e.g. you must explicitly capture and drop everything you don't want in the output string rather than printing only the things you do want, which is OK for the general case but does not work when you are trying to match multiple items per line, e.g.

# Try to extract the numbers found in each 'foo_x' occurence

# this prints the entire word each time, followed by the number
echo "foo1x foo2x foo3x" | string match -ar 'foo(\d)x'

# this prints the number only, but does not match more than once
echo "foo1x foo2x foo3x" | string replace -r '.*foo(\d)x.*' '$1'
# prints only the very last match `3`

You can hack around it by using string match -ar and constraining the pattern to not consume more than one match, e.g. in this case string replace -ar '\b\S*foo(\d)x\S*' '$1' but a) it is not always possible to do so, e.g. in the case of patterns spanning multiple words/values, and b) it is extremely awkward.

My suggestion is going with something like

echo "foo1x foo2x foo3x | string match -ar 'foo(\d)x' '$1'

or

echo "foo1x foo2x foo3x | string match -ar 'foo(\d)x' --print '$1' # or similar

to tell string match to output only a transformation of the match based off the parameter, rather than emitting all captures or the all input plus the captures.

[faho]

I think I would prefer an extension to string match -r along the lines of string replace -r with an additional parameter describing what to output, e.g.

This isn't possible. What should

string match -r 'foo (bar) baz' '$2'  "foo bar baz"

do? Anything after the first parameter is currently a "string" to act on, this can't be optional or it's ambiguous.

Anyway, I fail to see how --groups-only wouldn't be enough, this would just be

echo "foo bar baz" | string match -r --groups-only 'foo (bar) baz'

echo "foo1x foo2x foo3x" | string replace -r '.*foo(\d)x.*' '$1'

Depending on what you want,

echo "foo1x foo2x foo3x" | string replace -ar 'foo(\d)x' '$1'

would work, which yields one string "1 2 3".

echo "foo1x foo2x foo3x | string match -ar 'foo(\d)x' '$1'

is

echo "foo1x foo2x foo3x" | string match -ar --groups-only 'foo(\d)x'

which would yield "1", "2" and "3".

---

Anyway, I gotta get that branch cleaned up and merged.

[faho]

Alright, I just added string match --group-only. It's pretty straight-forward, just removes the entire match from consideration.

This manages to solve all examples people came up with here, as far as I see, so I don't see the need for further transformations. Of course we can still add that later.