New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plans for pcre2 support? #1335

Open
cpaelzer opened this Issue Sep 26, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@cpaelzer

cpaelzer commented Sep 26, 2018

Hi,
pcre2 was not mentioned commits nor on the mailing lists nor the issue tracker that now is mirrored to GH, unless I missed something.

pcre2 is available since early 2015 and projects could start considering it stable, but due to a change in the API it usually is more than just changing the include being used.
pcre (8.x) continues to exists, but as bugifx only stream and any new features/development are going into pcre2 (10.x).

Therefore Ubuntu would like to demote pcre in favor of pcre2 at some point, but Nmap (among other pkgs) would need to be ported to use pcre2.

Therefore I wanted to ask if there are any plans to do so (or not) in your project?
If there is none yet, please feel free to use this issue to track it.

@dmiller-nmap

This comment has been minimized.

Show comment
Hide comment
@dmiller-nmap

dmiller-nmap Sep 28, 2018

Thanks for suggesting this. The Nmap Project currently has no plans to update to PCRE2, but we have no immediate objections. However, these are our questions that would have to be answered before we commit to an upgrade:

  1. Will all of our tens of thousands of existing regular expressions match the same strings that they did under original PCRE?
  2. Does PCRE2 build and run on the platforms we need it to? Platforms officially supported by Nmap include GNU/Linux, Windows, Solaris, AIX, and FreeBSD, among others.
  3. Has performance been negatively impacted for our specific use case? We compile tens of thousands of regular expressions and then attempt to find the one that matches some untrusted input, for up to several thousand inputs in succession.

There may be other concerns, but these are the primary ones I can think of.

dmiller-nmap commented Sep 28, 2018

Thanks for suggesting this. The Nmap Project currently has no plans to update to PCRE2, but we have no immediate objections. However, these are our questions that would have to be answered before we commit to an upgrade:

  1. Will all of our tens of thousands of existing regular expressions match the same strings that they did under original PCRE?
  2. Does PCRE2 build and run on the platforms we need it to? Platforms officially supported by Nmap include GNU/Linux, Windows, Solaris, AIX, and FreeBSD, among others.
  3. Has performance been negatively impacted for our specific use case? We compile tens of thousands of regular expressions and then attempt to find the one that matches some untrusted input, for up to several thousand inputs in succession.

There may be other concerns, but these are the primary ones I can think of.

@cpaelzer

This comment has been minimized.

Show comment
Hide comment
@cpaelzer

cpaelzer Sep 28, 2018

Hi,
thanks for your answer @dmiller-nmap all concerns sound very reasonable.
First of all I'm not an pcre expert at all, just trying to understand/coordinate various projects position to prce2 from a Distributions perspective.
Therefore I'm trying to answer your concerns to my best knowledge, but do not really feel authoritative to do so- upstream pcre might be the right place to check these I'd think.

  1. AFAICT the API changed but not the Rules. There are new constants that can be added to modify the behavior of rules but I'd expect the rules to match the same. News and Changelog are meant to give a high level overview and I found nothing suggesting that rules would behave differently.
  2. Sorry I don't know, but again I have not found anything in upstreams statements that ould sugegst that pcre2 would be inferior in that regard. Via search engines I found pcre2 builds for all platforms you named, so I'd think yes - the same platforms are supported.
  3. Well given that any new development since 2015 is in pcre2 I'd hope that if anything it got better there.

As I said, I'm the middle man in this case.
I'm moore "trying to track your intention and status" than "trying to convince" you on this.

cpaelzer commented Sep 28, 2018

Hi,
thanks for your answer @dmiller-nmap all concerns sound very reasonable.
First of all I'm not an pcre expert at all, just trying to understand/coordinate various projects position to prce2 from a Distributions perspective.
Therefore I'm trying to answer your concerns to my best knowledge, but do not really feel authoritative to do so- upstream pcre might be the right place to check these I'd think.

  1. AFAICT the API changed but not the Rules. There are new constants that can be added to modify the behavior of rules but I'd expect the rules to match the same. News and Changelog are meant to give a high level overview and I found nothing suggesting that rules would behave differently.
  2. Sorry I don't know, but again I have not found anything in upstreams statements that ould sugegst that pcre2 would be inferior in that regard. Via search engines I found pcre2 builds for all platforms you named, so I'd think yes - the same platforms are supported.
  3. Well given that any new development since 2015 is in pcre2 I'd hope that if anything it got better there.

As I said, I'm the middle man in this case.
I'm moore "trying to track your intention and status" than "trying to convince" you on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment