Skip to content

Extract common regex compilation cache for reuse across regexp functions #23675

Description

@andygrove

Is your feature request related to a problem or challenge?

While reviewing #23540 (optimizing regexp_instr), it was noted that the new RegexCache introduced there overlaps with the existing compile_and_cache_regex helper in datafusion/functions/src/regex/mod.rs.

The version added in #23540 is a superset: in addition to the HashMap keyed by (pattern, flags), it memoizes the previous row's pattern (last) so that the common case of a literal pattern skips the hash lookup entirely on every row after the first. This gave a meaningful part of the ~40% speedup in that PR.

Today only regexp_instr benefits from this fast path. The other regex functions (regexp_count, regexp_match, regexp_replace) still use the plain compile_and_cache_regex HashMap lookup.

Describe the solution you'd like

Extract the RegexCache pattern from regexp_instr into datafusion/functions/src/regex/mod.rs as a shared, reusable cache, and migrate the other regex functions onto it so they get the same last-pattern memoization:

  • regexp_count
  • regexp_match
  • regexp_replace

This would consolidate the two caching implementations into one and let all regex functions skip per-row hashing for literal patterns.

Describe alternatives you've considered

Leave the two implementations separate. This keeps mod.rs unchanged but duplicates the caching logic and leaves the other functions without the faster path.

Additional context

Follow-on from #23540. Suggested by @alamb in review:

Maybe we could (as a follow on PR) extract the common regexp caching functionality -- I personally prefer this Cache pattern -- for use in the other functions with regular expressions

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions