Support for Antlr4 and PHP 7.0

[Mikulas]

Hi,
current php module supports 5.2 grammar which is officially not supported since 6 Jan 2011 (http://php.net/eol.php). This pull request adds module for antlr4, which is required for modern grammar definitions, and php 7.0 generator based on Ivan Kochurkin's open source grammar (https://github.com/antlr/grammars-v4/tree/master/php).

This works for me as it is, so I'm asking for comments before continuing any work. Are you interested in merging this? If so, I'll refactor the code and add tests.

[coveralls]

Coverage remained the same at 54.514% when pulling b2d2335 on Mikulas:php71 into 786aa9f on GumTreeDiff:develop.

[jrfaller]

Hi!

Thanks a lot for the PR, However, it seems that it is a PHP 5.6 grammar and not 7.x for instance I tried with function return type annotations and it seems to not be able to parse it. Am I right?

Cheers!

[Mikulas]

Oh that is correct, it is in fact 5.6 grammar. So far I've been testing simple trees such as <?php return 1+2; so I didn't run into that.

I have not found any 7.x compatible grammar for antlr 4.

As far as I know the only up-to-date parser is the internal one (https://github.com/nikic/php-ast) and userland php parser (https://github.com/nikic/PHP-Parser). I'll fall back to XML matching similar to python support discussed at #31.

Thank you :)

[jrfaller]

Agreed, these two libraries look nice!

You can look how we do it in the gen.c module which relies on an external parser as well.

[morandat]

I was looking again to your PR. Maybe you should split it in two, the part related to AntLR4 could still be useful.

[coveralls]

Coverage remained the same at 54.514% when pulling b2d2335 on Mikulas:php71 into 786aa9f on GumTreeDiff:develop.

[svanteschubert]

I have exchanged the PHP part with XML grammar from the ANTLR book and overtake the Java part of this patch to reuse your work, Mikulas.

But I have trouble with the Java part and need some help.

The resulting patch can be found here:
#61

[jrfaller]

Hi @Mikulas! BTW did you manage to feed a PHP AST into GumTree?

Cheers.

[Mikulas]

Hi @jrfaller, I ended up using the old PHP 5.6 (https://github.com/antlr/grammars-v4/tree/master/php) grammar for antlr4. I was mostly interested in automatic code merging on the AST level (as opposed to the traditional line based merging), and for that the old PHP version was sufficient.

Also back then a new parser from Microsoft was just released https://github.com/Microsoft/tolerant-php-parser. It was not stable enough a year ago so I chose not to use it; now it seems to be the best choice for a versioning algorithm, as some kind of syntax error recovery is very much welcome.

Other than that, JetBrains have a very good tolerant parser for their IDE (PhpStorm), but it's proprietary. That's all the PHP parsers I know of.

[jrfaller]

Hi thanks for the information, we will look into the microsoft parser but it seems to be a pain to install ;) but definitely seems doable.

Cheers.