preg_split(): Compilation failed: escape sequence is invalid

[erlangsec]

I get the error message above, which looks like a regex compliation bug related to PHP 7.3, originating from:
/vendor/zbateson/mail-mime-parser/src/Header/Consumer/AbstractConsumer.php:187

According to this resource, e.g. "the hyphen now needs to be escaped" (if this could be a relevant case).

See also PHP 7.3 migration guide for PCRE:
https://wiki.php.net/rfc/pcre2-migration#backward_incompatible_changes

[zbateson]

Hi @erlangsec --

Could you give more details on where that may be happening, or even more helpful would be a test case? Currently it seems all tests are passing in 7.3 for version 1.1.5... but maybe I'm missing a test somewhere. I had a quick look around at the generated patterns as well but couldn't find any hyphens.

[erlangsec]

Thanks for asking, the bug is triggered by any Received header, as far as I can tell, including the one from your original tests:

Received: from LeComputer (blah.host) by MyComputer ([1.2.2.2]) with ESMTP (TLS BLAH) ID 123; Wed, 17 May 2000 19:08:29 -0400

Apparently, this header is also treated with some special parsing patterns (which makes sense).

[erlangsec]

I've found that the character \A does not compile, in /src/Header/Consumer/Received/GenericReceivedConsumer.php line 126.

The error is resolved by switching from PHP7.3 to PHP7.2. Or by removing/replacing the character.

[erlangsec]

While I do not have any official source for this theory, it looks like since \A cannot be repeated, it cannot be contained in a group which characters can be repeated, and this is validated more strictly in PCRE2/PHP7.3.

Proposed fix in pull request #93.

[zbateson]

Excellent, thanks for that @erlangsec

I'll review your change and aim to release a fix this weekend... can't for the life of me think of why I grouped \A like that, so will have a look around.

Can you tell me if, without that fix, a unit test is failing for you and which one, and if your fix fixes it for you as well? It seems Travis on PHP 7.3 isn't erroring out, and I'm wondering if you may know why:

https://travis-ci.org/zbateson/mail-mime-parser/jobs/573601308

Thanks again!

[erlangsec]

Without proposed fix:
............................................................... 63 / 452 ( 13%) .......EEEEEE.EEEEEEEEEE..................EE..E................ 126 / 452 ( 27%) ...................................................EEEEE....... 189 / 452 ( 41%) ............................................................... 252 / 452 ( 55%) ............................................................... 315 / 452 ( 69%) ............................................................... 378 / 452 ( 83%) ............................................................... 441 / 452 ( 97%) ........... 452 / 452 (100%)

There were 24 errors:

1) ZBateson\MailMimeParser\Header\Consumer\Received\DomainConsumerTest::testConsumeParts preg_split(): Compilation failed: escape sequence is invalid in character class at offset 49
2) ZBateson\MailMimeParser\Header\Consumer\Received\GenericReceivedConsumerTest::testConsumeTokens preg_split(): Compilation failed: escape sequence is invalid in character class at offset 49

and so on ...

But with fix, no errors:
............................................................... 63 / 452 ( 13%) ............................................................... 126 / 452 ( 27%) ............................................................... 189 / 452 ( 41%) ............................................................... 252 / 452 ( 55%) ............................................................... 315 / 452 ( 69%) ............................................................... 378 / 452 ( 83%) ............................................................... 441 / 452 ( 97%) ........... 452 / 452 (100%)

My environment:
PHP 7.3.8-1+ubuntu18.04.1+deb.sury.org+1 (cli) (built: Aug 7 2019 09:52:12) ( NTS ) Copyright (c) 1997-2018 The PHP Group Zend Engine v3.3.8, Copyright (c) 1998-2018 Zend Technologies with Zend OPcache v7.3.8-1+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies

[zbateson]

Released in 1.1.6. Thanks for your help on this one!