Multiple matches for named capture group

[spease]

Hi,
I was wondering if there's any documentation wrt to named capture groups that are expected to have multiple matches, and/or how to handle this scenario? Thanks.

[BurntSushi]

@spease I don't understand the question. Could you please be more specific? If possible, please include examples and code that you've tried writing, including its actual output and its expected output. It might also help to point to other regex engines that have this feature that you know how to use.

[varkor]

If I understand the question, some regex engines allow you to have multiple named capture groups in a single regex: for example a(?<digit>[0-5])|b(?<digit>[4-7]), which matches either "a" followed by 0 to 5, or "b" followed by 4 to 7, and stores the matched digit in digit. There's a bit of flexibility here when multiple groups match (although often they'll be explicitly non-overlapping) — a reasonable choice is to pick the first group that matches.

[BurntSushi]

OK, thanks for clarifying. This regex library doesn't support multiple capture groups with the same name. You can see your example here: https://play.rust-lang.org/?gist=c83fb1437a539c6f98cd90591965fc78&version=stable

Again, it's not clear if you knew this or not. Are you asking for this feature to exist? Are you asking for a work around? I don't know. Please be more specific. If you're asking for a work around, then please describe the specific problem you're trying to solve.

[ethanpailes]

@varkor, What would the behavior be for a situation like (?<digit>[0-9])(?<digit>[0-9]) on 12? I think taking the rightmost match would be the easiest to implement, but I'm curious about what other engines do.

[BurntSushi]

(To be clear, the reason why the regex engine disallows multiple capturing groups with the same name is precisely because of these sorts of questions. There may be a set of sensible answers for all of them, but a solid specification that includes answers for all of them along with implementation paths would be a prerequisite to supporting this functionality. Another prerequisite is justifying its existence, which I don't think has been done yet.)

[varkor]

@ethanpailes: The behaviour isn't consistent between engines. I think if the feature was implemented,the behaviour would be fairly unimportant (overlapping cases should be discouraged), as long as it was clearly documented. But I agree with @BurntSushi that the ambiguity of the semantics makes it unattractive as a feature (unless perhaps there were some way to statically guarantee that none of the case overlapped, though this seems nontrivial). You can always achieve what you want without multiple groups of the same name.

[spease]

@BurntSushi Sorry for the delay. I'm actually referring to a capture group being matched multipe times. The core processing code is below, though

                    phrases
                        .iter()
                        .filter_map(|phrase| {
                            let capture_names = phrase.capture_names();
                            phrase.captures(&*contents.borrow()).map(|captures| {
                                capture_names
                                    .filter_map(|name_opt| {
                                        name_opt.and_then(|name| {
                                            captures.name(name).map(|value| {
                                                (name.to_owned(), value.as_str().to_owned())
                                            })
                                        })
                                    })
                                    .collect::<Vec<(String, String)>>()
                            })
                        })
                        .flat_map(|x| x)
                        .collect::<Vec<(String, String)>>()

where phrases are generated with:

struct MacroReplacer;

impl regex::Replacer for MacroReplacer {
    fn replace_append(&mut self, caps: &regex::Captures, dst: &mut String) {
        let mut submatches = caps.iter().skip(1);

        write!(
            dst,
            "(?P<{}>{})",
            submatches.next().unwrap().unwrap().as_str(),
            match submatches.next().unwrap().unwrap().as_str() {
                "number" => r"[-+]?[0-9]*\.?[0-9]+",
                "word" => r"\w+",
                "words" => r"[\w\s]+",
                ty => panic!(format!("Unknown type {}", ty)),
            }
        ).unwrap();
    }
}

    let phrases: Vec<Regex> = BufReader::new(std::fs::File::open(&opt.phrase_filepath)?)
        .lines()
        .map(|line_result| {
            line_result.map_err(|e| e.into()).and_then(|line| {
                Regex::new(&line_re.replace_all(&line, MacroReplacer {})).map_err(|e| e.into())
            })
        })
        .collect::<Result<Vec<Regex>>>()?;

The ultimate goal being to generate a vector of all the matches with the key being the capture group name, and the value being the capture group value.

The problem I ran into was that this only seems to match one instance of the capture group.

The general concept here is a sort of reverse template that's easily usable even if one isn't familiar with regexes.

[BurntSushi]

Can you describe the problem you are trying to solve at a higher level please? That is, don't mention anything about regular expressions it capture groups in your problem description. Otherwise it is too hard for me to follow what you are saying. (I will take a closer look at the code you've shared, but that will have to wait.)

…

On Dec 25, 2017 5:44 PM, "spease" ***@***.***> wrote: @BurntSushi <https://github.com/burntsushi> Sorry for the delay. I'm actually referring to a capture group being matched multipe times. The core processing code is below, though phrases .iter() .filter_map(|phrase| { let capture_names = phrase.capture_names(); phrase.captures(&*contents.borrow()).map(|captures| { capture_names .filter_map(|name_opt| { name_opt.and_then(|name| { captures.name(name).map(|value| { (name.to_owned(), value.as_str().to_owned()) }) }) }) .collect::<Vec<(String, String)>>() }) }) .flat_map(|x| x) .collect::<Vec<(String, String)>>() where phrases are generated with: struct MacroReplacer; impl regex::Replacer for MacroReplacer { fn replace_append(&mut self, caps: &regex::Captures, dst: &mut String) { let mut submatches = caps.iter().skip(1); write!( dst, "(?P<{}>{})", submatches.next().unwrap().unwrap().as_str(), match submatches.next().unwrap().unwrap().as_str() { "number" => r"[-+]?[0-9]*\.?[0-9]+", "word" => r"\w+", "words" => r"[\w\s]+", ty => panic!(format!("Unknown type {}", ty)), } ).unwrap(); } } let phrases: Vec<Regex> = BufReader::new(std::fs::File::open(&opt.phrase_filepath)?) .lines() .map(|line_result| { line_result.map_err(|e| e.into()).and_then(|line| { Regex::new(&line_re.replace_all(&line, MacroReplacer {})).map_err(|e| e.into()) }) }) .collect::<Result<Vec<Regex>>>()?; .iter() .filter_map(|phrase| { let capture_names = phrase.capture_names(); phrase.captures(&*contents.borrow()).map(|captures| { capture_names .filter_map(|name_opt| { name_opt.and_then(|name| { captures.name(name).map(|value| { (name.to_owned(), value.as_str().to_owned()) }) }) }) .collect::<Vec<(String, String)>>() }) }) .flat_map(|x| x) .collect::<Vec<(String, String)>>(), The ultimate goal being to generate a vector of all the matches with the key being the capture group name, and the value being the capture group value. The problem I ran into was that this only seems to match one instance of the capture group. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#432 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAb34t9xjSg44uQwBA8xdRzV25P7PPwgks5tECVPgaJpZM4RJeOL> .

[spease]

Sure. I'd like to be able to define an inverse template for data mining text files, using a form

{name:type}

for the variables. For instance:

My name is {name:word}, and I am {age:number} years old.

However in some cases there might be multiple matches for the same thing:

EDUCATION: {school:words}

EDUCATION: Harvard
EDUCATION: Massachusetts Insitute of Technology

In this case I'd like to get a list of each matching item. If there is only one matching item, it's OK if that's a list too. Eg for the immediately prior example:

{"school": ["Harvard", "Massachusetts Insitute of Technology"] }

[BurntSushi]

@spease Why doesn't find_iter or captures_iter work for you? Have you read this crate's documentation? I imagine you want a type like HashMap<String, Vec<String>>, but I don't see that anywhere in your code.

[spease]

I'm not sure why captures_iter wouldn't work - I thought I looked at it and find_iter, but I remembered it as not having a way to get the name of the match, which it does. The code that stores the matches for a file is not present in the excerpts I posted here but is present later in the program. Let me look into your suggestions and get back to you.

…

On Fri, Dec 29, 2017 at 3:43 AM, Andrew Gallant ***@***.***> wrote: @spease <https://github.com/spease> Why doesn't find_iter or captures_iter work for you? Have you read this crate's documentation? I imagine you want a type like HashMap<String, Vec<String>>, but I don't see that anywhere in your code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#432 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACsb9EsJDg1nFLLVIv7Jr3B2O6QJKugFks5tFNBwgaJpZM4RJeOL> .

-- - Steven

[BurntSushi]

Closing due to inactivity.