Different preg_match result with `-d pcre.jit=0`

[mvorisek]

Description

related with PHP-CS-Fixer/PHP-CS-Fixer#6997

How to reproduce:

1. clone https://github.com/PHP-CS-Fixer/PHP-CS-Fixer.git (latest master or 3.17 tag)
2. run composer update
3. run php -d pcre.jit=0 vendor/phpunit/phpunit/phpunit --filter TypeExpressionTest
4. the tests will fail, and notice, with -d pcre.jit=1 the tests are passing

Expected result:

pcre.jit config should have no effect on the preg_match result

The problematic regex is probably https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/blob/v3.17.0/src/DocBlock/TypeExpression.php#L32 - is there any easy workaround until php-src/PCRE is fixed?

PHP Version

PHP 7.4 - 8.2

Operating System

Windows and Unix

[damianwadley]

Huh. It's backwards: normally these sorts of bugs happen with jit=1 and turning it off allows the regex to work...

That's a gigantic regex. Can you boil it down to something manageable that still reproduces?

And either way, does it also happen with the pcretest utility? If so then it's an upstream bug, which is my suspicion, and not PHP itself.
https://www.pcre.org/original/doc/html/pcretest.html

[damianwadley]

is there any easy workaround until php-src/PCRE is fixed?

Sure: turn on JIT 😉

Otherwise, maybe? You could experiment with the regex - add/remove optimizations, that sort of thing - to see if you can trigger some different internal code paths to bypass wherever the bug is. It'd be easier to do that if you could narrow down the source of the problem first, of course.

And there's always the option of breaking it down into smaller regexes... I mean really, that's 100+ lines of (formatted) regex in there. May or may not help, but it might be a nice gain regardless.

[Wirone]

Sure: turn on JIT 😉

In my case there's PCRE JIT Support => not compiled in so I can't enable it (asdf's PHP on M1 Mac). I can workaround it with Docker setup, but I just wanted to point that it's not always possible to use PCRE JIT.

[iluuu1994]

The issue is indeed cause by JIT, not the other way around. Simpler reproducible case:

$regex = '
    (?<types>
        (?:
            (?:\{ (?&types) \})
            | (a)
        )
        (\*?)
    )
';

var_dump(preg_match('{^' . $regex . '$}x', '{a}', $matches), $matches);

This is the correct output, with pcre.jit=0.

int(1)
array(5) {
  [0]=>
  string(3) "{a}"
  ["types"]=>
  string(3) "{a}"
  [1]=>
  string(3) "{a}"
  [2]=>
  string(1) "a"
  [3]=>
  string(0) ""
}

This is the incorrect output, with pcre.jit=1.

int(1)
array(5) {
  [0]=>
  string(3) "{a}"
  ["types"]=>
  string(3) "{a}"
  [1]=>
  string(3) "{a}"
  [2]=>
  string(0) ""
  [3]=>
  string(0) ""
}

Note the missing capture for group 2.

Looking at the offsets, without JIT:

0 - 3
0 - 3
1 - 2
3 - 3

With JIT:

0 - 3
0 - 3
18446744073709551615 - 18446744073709551615
3 - 3

So, the issue seems to originate from PCRE. I'll check if the issue was already fixed/reported upstream.

Edit: Things are a bit more complicated. Removing the (\*?) captures 2 groups instead of the original 4, which makes no sense to me.

[mvorisek]

Here is online repro - https://3v4l.org/W4UkB, problem is present somewhere between PCRE >8.41 and <= 10.32 versions.

[iluuu1994]

Yeah so it does seem JIT is the odd one out after all, although the output doesn't make sense to me.

[remicollet]

problem is present somewhere between PCRE >8.41 and <= 10.32 versions.

• 8.x is libpcre used by PHP <= 7.2
• 10.x is libpcre2 used by PHP >= 7.3