Python performance issue while parsing ECMAScript

[renatahodovan]

I've tried to parse one of the githubs JS backend files with the "official" ECMAScript grammar. Surprisingly, the parse took only 2 minutes in Java but more than 4 hours with Python3 (even with the recently added performance fix too).
The test case is undoubtedly large and - as @ericvergnaud said earlier - Python is 20 - 30 times slower than Java. But in this case the slow-down factor is more than 120 so there might be something else in the background too.
I'm trying to figure out the reason but also reporting the issue hoping that someone else can solve it faster.

[parrt]

@renatahodovan could this be related to #994 ? Are you trying to parse very large expressions with a left recursive rule perhaps? Or maybe it is a hash code issue that JavaScript had #1408? @ericvergnaud any ideas?

[renatahodovan]

@parrt Yes, it might be related to recursive rules - it was my first thought too -, although I don't know how to fix it. On the other hand, Python has its own hash function - unlike JavaScript - so I don't think that we could outperform that with a new implementation.
I've attached a slightly minimized version of my original test case. It's now ~20KB of valid ECMAScript code. The parsing takes ~50sec for me with the latest Python3 target and ~1sec with Java (not like 50 sec is unbearable but unfortunately this difference scales up and in case of a bigger real life test case 2 minutes vs. 4 hours is inconvenient).
The attachment also contains a cProfile entry which might help understanding what happens.

[parrt]

If Java target was fast on it, then it's not the new optimization; likely hash function issue?

[ghost]

I tried parsing a 4 megabyte Javascript file with the python 3 backend on ANTLR 4.7 and its extremely slow. The python slimit Javascript parsing library (ply based) takes 15 seconds to parse the file but I hit a bug in it that I don't know how to fix, so I decided to try ANTLR. I haven't really seen anything else so things seem pretty hopeless, writing my own ECMAScript parser doesn't seem very feasible...

With respect to how much slower it is, I don't know because its still running but its taken 40 minutes of cpu time so far...

Thanks for any input.

Edit: I can't post the js file publicly because it's part of a game. I'm trying to mod it.

[ericvergnaud]

Hi The place for support is the google discussion group Since you're looking for speed, using Python is a challengeable idea in the first place. Why not use Java? Eric Envoyé de mon iPhone

…

Le 7 août 2017 à 01:00, 22459 ***@***.***> a écrit : I tried parsing a 4 megabyte javascript file with the python 3 backend on antlr 4.7 and its extremely slow. The python slimit javascript parsing library (ply based) takes 30 seconds to parse the file but I hit a bug in it that I don't know how to fix, so I decided to try ANTLR. I haven't really seen anything else so things seem pretty hopeless, writing my own ecmascript parser doensn't seem very feasible... With respect to how much slower it is, I don't know because its still running but its taken 40 minutes of cpu time so far... Thanks for any input. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.