Consider migrating away from JavaCC

[ftomassetti]

The syntax is identical and so I would expect this should require minimal effort.

The benefit would be a better maintained project.

See tulipcc/ParserGeneratorCC#17

Some metrics: JavaCC had no commits in the last 10 months. That is roughly the age of ParserGeneratorCC which has 900 commits. For this reason it seems to me a project that could offer better support and perhaps better features in the future.

What do you guys think?

[matozoid]

I think there is a new version of JavaCC being worked on, and that is our upgrade path. I'd rather have a cool new version than a pretty but outdated fork. Moving to ANTLR would be preferred though.

[ftomassetti]

Should I say that we are three options for V4 then?

1. Keep JavaCC
2. Move to ParserGeneratorCC
3. Move to ANTLR

Should we discuss these? Any other option?

[ftomassetti]

Or add your new GLR parser generator as option 4

[matozoid]

GLL parser :-) That ought to be another discussion since that will probably perform nowhere near ANTLR and JavaCC.

I wouldn't put any real work into anything but ANTLR.

[ftomassetti]

Moving to ANTLR would mean having to use a runtime and it would be significant work.
Moving to ParserGeneratorCC would be much less work.

I can leave with both but for ANTLR I would expect the work not to be trivial mostly because ANTLR generate AST classes so we could not use anymore our AST classes. Sure, unless we change the ANTLR generator, which technically is possible (I have done exactly that to create the Kotlin target). If you are interested in that we can explore it

[ftomassetti]

Funny thing: if we are using a version of ph-javacc-maven-plugin 4+ we have already migrated to ParserGeneratorCC without realizing it as the plugin was using JavaCC in version 3- and ParserGeneratorCC since version 4+

Update: nope, we are using version 3

[ftomassetti]

I tried in a branch to just change the version of ph-javacc-maven-plugin to version 4.0.3 so that it uses ParserGeneratorCC instead of JavaCC. It works like a charm: no changes needed. So the effort to migrate to ParserGeneratorCC is extremely low

[matozoid]

It's not an option: the upgrade path for JavaCC is JavaCC, not that fork. I expect major new features from JavaCC, and moving to a fork could make that harder. What would it even add? It's nice that the code is refactored, but when the JavaCC is out they can start all over since they failed to work with the project owners. Yes, I know they are impossibly secretive, but the fact remains that the code bases are split. And the fork guy is really nice, but I really don't know why he's wasting his time on it.

[ftomassetti]

I understand, I will close this issue. We could open another one if we want to discuss migrating to ANTLR, as the GLL parser you stated could have performance issue at this moment.

Out of curiosity, how come you expect new features from JavaCC? In my impression there is no development going on and no commits for almost one year now

[matozoid]

Well, I honestly can't find it anymore, I just looked through the issue tracker there. A version 8 is mentioned, and I remember him wanting to implementing a completely new algorithm or so. He was telling some other guy to stop refactoring stuff because it was all going away anyway.

Indeed there's not much going on there anymore. What do you say we put a time limit on it: if there is still no sign of life one year from now, we go for the fork which has a way too long name so I keep calling it "the fork." We can already move to it, but we shouldn't use specific features until one year from now.

[ftomassetti]

It seems reasonable to me to re-evaluate in one year. Perhaps at that point your parser will also be a candidate. After all my concerns are more about the future state of the project than the differences in features we have right now. Just do not updat the ph-javacc-maven-plugin or we will migrate without even realize it :)

[matozoid]

Right, let's take action around September 2019 then!

[malteskoruppa]

Here's just my 2 cents. :-)

I agree it would not make a lot of sense to move to the fork, at least for now. The project looks nice, but it's unclear how well it will establish itself as a true alternative to JavaCC (I'm rather sceptical) that will be well-maintained for a long time to come. It could be that the only maintainer decides to stop working on it a few months from now. The point is, if JavaParser moves to ParserGeneratorCC, then JavaParser depends on a little-known parser generator maintained by a single person, and that seems like a very thin thread to me.

Of course, if things change and ParserGeneratorCC becomes a well-known and widely-used tool maintained by a sizeable team, it's a different story.

ANTLR seems like the best alternative in the long run, but of course, it's a problem that the effort required for migrating to it is extremely high.

So yes... re-evaluating things a year from now is probably the best thing to do right now. :-)

[ftomassetti]

@malteskoruppa I would agree that I would not move from an established, well-maintained project to one that is uncertain. However my impression is that JavaCC is not maintained at all. This is way I am even considering looking for alternatives

[malteskoruppa]

Ok, I haven't worked with JavaCC so far, and I can't really judge how well it is maintained. It's true that the last commit was made 10 months ago. It's certainly a good idea to look for alternatives, if only to know what's out there. It's just that, to me, moving to an almost unknown, very young fork maintained by a single person seems to be a bit of a leap of faith.

ANTLR would be truly awesome. This is a tool that has been around for 30 years and is still actively maintained. :-)

[ftomassetti]

It's just that, to me, moving to an almost unknown, very young fork maintained by a single person seems to be a bit of a leap of faith.

Maintained by a single person wins against unmaintained :)

ANTLR would be truly awesome. This is a tool that has been around for 30 years and is still actively maintained. :-)

I use ANTLR a lot (I also wrote a Kotlin target for it) however there are two issues:

1. It requires a runtime and currently JP core has zero dependencies
2. It generates ugly AST classes, while with JavaCC we can generate our own. Now, we could potentially change that if we want to create our own alternative Java target, which would be doable but it would a significant effort
3. In any case migrating to ANTLR would be a lot of work

[malteskoruppa]

Maintained by a single person wins against unmaintained :)

Sure, I get your point. :) But really, either alternative doesn't look particularly attractive to me. :/

[malteskoruppa]

It requires a runtime and currently JP core has zero dependencies

Interesting. I wasn't aware of that! 😲 Intuitively, I thought that JP core depends on JavaCC. But from a quick look, it seems like JavaCC simply generates Java source files from a .jj file, which are then subsequently used by JP core. So the JP core itself does not depend on JavaCC. Is that correct?

While that's a neat decoupling of the tools involved in the toolchain, in a sense I think you could nevertheless say that JP core depends on JavaCC: Without JavaCC, there would be no JP core (or at least, without JavaCC, the JP core couldn't be easily updated). It's just that JP "depends" on a plugin and not on a compile-time dependency. While that may be a lesser form of dependency (so to speak), JavaCC is still a requirement for bootstrapping the JP core.

Another interesting point which hasn't been explicitly mentioned so far: The (only) maintainer of the ph-javacc-maven-plugin is the same guy who is also the (only) maintainer of ParserGeneratorCC. So things seem to be moving in the direction of ParserGeneratorCC anyway. Therefore, I agree with what you both already said: JP might as well move to ParserGeneratorCC, it simply shouldn't use any specific features in case it wants to move back to JavaCC in the future.

[matozoid]

I just noticed that JavaCC's development work is done on branches, not on master. You hardly ever see activity there (although there is now a new release).