Skip to content

regex_match drops every capture after a non-participating group, and hard-caps at 15 groups #629

Description

@InauguralPhysicist

Finding

regex_match stops emitting captures at the first group that didn't participate, and silently ignores groups past 15. Both break positional indexing, with rc=0.

src/builtins.c:1594-1608:

regmatch_t matches[16];
if (regexec(&re, str, 16, matches, 0) != 0) { ... }
...
for (int i = 0; i < 16 && matches[i].rm_so >= 0; i++) {

POSIX sets rm_so = -1 for a group that didn't participate. Having that test in the loop condition turns "this group is empty" into "stop emitting groups".

Reproduction

print of ["(x)?(a)(b) on 'ab' ->", regex_match of ["ab", "(x)?(a)(b)"]]
print of ["(a)(b)     on 'ab' ->", regex_match of ["ab", "(a)(b)"]]
["(x)?(a)(b) on 'ab' ->", ["ab"]]            <- matched, but ZERO groups returned
["(a)(b)     on 'ab' ->", ["ab", "a", "b"]]  <- control: works

Python on the same pattern: re.match('(x)?(a)(b)', 'ab').groups()(None, 'a', 'b').

The match succeededa and b did participate — but one optional group ahead of them deletes both from the result. result[2] is then out of range rather than wrong-but-present.

A 17-group pattern returns 16 elements (full match + 15 groups); group 16 onward vanishes.

Invariant violated

docs/BUILTINS.md:111 specifies regex_match of [s, pattern][full_match, group1, ...]. On a successful match, group n must be at index n. Any optional group anywhere in a pattern silently breaks that for every capture after it — and (x)? is not an exotic pattern.

Not load-bearing

Every current use is the easy shape — all groups participate, count ≤ 2: tests/test_regex.eigs:5,37, tests/test_raise_on_bad_input.eigs:207-208 (indexes only [0]). lib/eigen.eigs:1773 re-exports the builtin. Nothing in tests/, docs/, lib/, or examples/ depends on the early break or the 16-cap. Textbook "works only because the examples are too easy".

Fix

Size the array from the compiled pattern and move the participation test into the loop body:

size_t ng = re.re_nsub + 1;
regmatch_t *matches = xmalloc(ng * sizeof(regmatch_t));
...
for (size_t i = 0; i < ng; i++) {
    if (matches[i].rm_so < 0) { list_append_owned(result, make_null()); continue; }
    ...
}

Choose null vs "" for a non-participating group deliberately and pin it in docs/BUILTINS.md — null is the more honest encoding. The heap allocation also drops a fixed stack array, consistent with the repo's no-big-by-value-arrays-in-recursive-paths rule.

Test to add

regex_match of ["ab", "(x)?(a)(b)"] → length 4, [1] null, [2] == "a", [3] == "b"; plus a 17-group pattern asserting length 18.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions