vre: Migrate to pcre2 #3635
Merged
vre: Migrate to pcre2 #3635
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merged
|
bugwash: OK, but I'll see what was left behind in the documentation before merging. |
Now that VRE is the only regular expression API we use, we can migrate its backend to pcre2. The existing 'pcre_*' parameters are also renamed to reflect this migration, and 'pcre_match_limit_recursion' gets special treatment and is renamed to pcre2_depth_limit. This creates an additional API breakage in VRE: the `match_recursion` field in `struct vre_limits` is also renamed. One last breakage is the removal of `VRE_has_jit` used by only one undocumented varnishtest feature, and the pcre_jit feature is only used by one test case that no longer fails. The pcre jit compilation feature was broken anyway: sealing it at compile time will not reflect what VRE actually links to. Once we have a test case needing the jit feature, we can introduce a better API for that check. There is one outstanding performance problem, the ovector that was previously allocated on the stack now needs to be allocated from the heap. It might be possible to implement a pcre2 context to fix that or maybe pool them, but for now we have heap allocations on the critical path. The VRE_sub() function makes sure to make a single ovector allocation (technically a pcre2_match_data allocation) since it's the only one guaranteed to loop on a single regular expression for the `regsuball()` use case. On the documentation front, the SmartOS installation instructions are hidden for lack of a pcre2 package. Closes varnishcache#3616 Closes varnishcache#3559
Dridi
added a commit
to Dridi/oss-fuzz
that referenced
this pull request
Jul 12, 2021
DavidKorczynski
pushed a commit
to google/oss-fuzz
that referenced
this pull request
Jul 16, 2021
* varnish: Migration to pcre2 See varnishcache/varnish-cache#3635 * varnish: Static linking to pcre2 * varnish: Only build the fuzzer Static linking of pcre2 fails the libvarnishapi build, not needed by the ESI fuzzer. That's also less time wasted building irrelevant components. We can figure out why libvarnishapi fails to build in this environment later. * varnish: Missing build step
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Now that VRE is the only regular expression API we use, we can migrate its backend to pcre2. The existing 'pcre_*' parameters are also renamed to reflect this migration, and 'pcre_match_limit_recursion' gets special treatment and is renamed to pcre2_depth_limit.
This creates an additional API breakage in VRE: the
match_recursionfield instruct vre_limitsis also renamed. One last breakage is the removal ofVRE_has_jitused by only one undocumented varnishtest feature, and the pcre_jit feature is only used by one test case that no longer fails.The pcre jit compilation feature was broken anyway: sealing it at compile time will not reflect what VRE actually links to. Once we have a test case needing the jit feature, we can introduce a better API for that check.
There is one outstanding performance problem, the ovector that was previously allocated on the stack now needs to be allocated from the heap. It might be possible to implement a pcre2 context to fix that or maybe pool them, but for now we have heap allocations on the critical path. The VRE_sub() function makes sure to make a single ovector allocation (technically a pcre2_match_data allocation) since it's the only one guaranteed to loop on a single regular expression for the
regsuball()use case.Closes #3616
Closes #3559
Only the last commit matters here, the rest comes from #3630.