Skip to content

fix: handle empty patterns in regexp_instr - #24054

Merged
neilconway merged 2 commits into
apache:mainfrom
iamhaseebn:fix-regexp-instr-empty-pattern
Aug 4, 2026
Merged

fix: handle empty patterns in regexp_instr#24054
neilconway merged 2 commits into
apache:mainfrom
iamhaseebn:fix-regexp-instr-empty-pattern

Conversation

@iamhaseebn

@iamhaseebn iamhaseebn commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

PostgreSQL treats empty and other zero-width regular-expression matches as valid character-boundary matches. Before this change, regexp_instr returned 0 before compiling an empty pattern, excluded the terminal boundary from its start-position mapping, and returned 0 instead of propagating NULL when the input was NULL and the pattern was empty.

What changes are included in this PR?

  • Let the regex implementation evaluate empty patterns instead of returning early.
  • Map the 1-based start position to UTF-8 byte boundaries including the terminal boundary.
  • Preserve NULL propagation when the input is NULL and the pattern is empty.
  • Add unit coverage for empty patterns and potentially zero-width patterns across Utf8, LargeUtf8, and Utf8View arrays.
  • Add SQL logic coverage for terminal zero-width matches and empty-pattern NULL input.

Are these changes tested?

Yes. The following checks pass on the current head:

  • cargo fmt --all -- --check
  • cargo test -p datafusion-functions regexpinstr::tests::test_regexp_instr
  • cargo test --profile=ci --test sqllogictests -- regexp_instr.slt
  • cargo clippy -p datafusion-functions --tests -- -D warnings

The implementation commit was also verified with the repository-prescribed extended workspace suite and cargo test --profile ci -p datafusion-cli; the review follow-up changes add tests only.

Are there any user-facing changes?

Yes. regexp_instr now returns PostgreSQL-compatible positions for empty and potentially zero-width patterns, including at the terminal character boundary, and returns NULL for NULL input with an empty pattern. This does not change the public API.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Aug 2, 2026

@neilconway neilconway left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @iamhaseebn ! Overall this looks good: the behavior change is in the right direction, and the implementation is sound.

It seems like this PR also changes behavior for two other cases, not mentioned in the PR description:

  • Patterns that might be empty width, like regexp_instr('abc', 'x*', 4)
  • Empty patterns on NULL input, like regexp_instr(NULL, '')

Both changes seem to make our behavior more consistent with PG, so I think they are good, but can you update the PR description and ensure there is test coverage for these cases?

@@ -479,7 +482,7 @@ mod tests {
fn test_case_sensitive_regexp_instr_nulls() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yours, but this function seems misnamed (doesn't involve NULLs). This seems redundant with other tests anyway, so we can probably just remove this function.

@iamhaseebn

Copy link
Copy Markdown
Contributor Author

Thanks for calling these out. I updated the PR description and added regression coverage for both cases:

  • a potentially zero-width x* match at the terminal boundary across Utf8, LargeUtf8, and Utf8View arrays;
  • NULL input with an empty pattern across the scalar string representations and the SQL logic test.

The focused unit test, SQL logic test, formatting check, and targeted Clippy check pass. The update is in 71fbe61.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.72131% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.87%. Comparing base (66c3840) to head (71fbe61).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/functions/src/regex/regexpinstr.rs 96.72% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24054      +/-   ##
==========================================
+ Coverage   80.86%   80.87%   +0.01%     
==========================================
  Files        1101     1101              
  Lines      375446   375816     +370     
  Branches   375446   375816     +370     
==========================================
+ Hits       303592   303954     +362     
+ Misses      53761    53754       -7     
- Partials    18093    18108      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@neilconway
neilconway added this pull request to the merge queue Aug 4, 2026
Merged via the queue into apache:main with commit f248f45 Aug 4, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PostgreSQL compatibility: regexp_instr with an empty pattern should return 1

3 participants