Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

formats.js won't parse with js2py #2669

Closed
Antar1011 opened this issue Aug 4, 2016 · 8 comments
Closed

formats.js won't parse with js2py #2669

Antar1011 opened this issue Aug 4, 2016 · 8 comments

Comments

@Antar1011
Copy link
Contributor

I'm using js2py to synchronize PS data (banlists, aliases, etc.) with my stats projects (esp. Onix), and currently my script which scrapes formats.js breaks. It worked as of a few weeks ago, so I'm wondering what might have changed to break it, and whether it's something that's absolutely needed. Here's my traceback:

>>> blah = scrapers.scrape_formats()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/antar/Programming/Onix/onix/scrapers.py", line 210, in scrape_formats
    raw_data = json.loads(_scrape(url, entry))
  File "/home/antar/Programming/Onix/onix/scrapers.py", line 56, in _scrape
    json_string = js2py.eval_js(prerun+javascript+postrun)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/evaljs.py", line 110, in eval_js
    return e.eval(js)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/evaljs.py", line 172, in eval
    self.execute(code, use_compilation_plan=use_compilation_plan)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/evaljs.py", line 167, in execute
    exec(compiled, self._context)
  File "<EvalJS snippet>", line 4, in <module>
  File "<EvalJS snippet>", line 3, in PyJs_LONG_0_
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/base.py", line 835, in __call__
    return self.call(self.GlobalObject, args)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/base.py", line 1281, in call
    return Js(self.code(*args))
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/host/jseval.py", line 16, in Eval
    py_code = translate_js(code.to_string().value, '')
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/translator.py", line 62, in translate_js
    parsed = parser.parse(js) # js to esprima syntax tree
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2862, in parse
    program = self.parseProgram();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2833, in parseProgram
    body = self.parseScriptBody()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2822, in parseScriptBody
    statement = self.parseStatementListItem();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1983, in parseStatementListItem
    return self.parseStatement();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2568, in parseStatement
    expr = self.parseExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1949, in parseExpression
    expr = self.isolateCoverGrammar(self.parseAssignmentExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1126, in isolateCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1939, in parseAssignmentExpression
    right = self.isolateCoverGrammar(self.parseAssignmentExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1126, in isolateCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1916, in parseAssignmentExpression
    expr = self.parseConditionalExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1817, in parseConditionalExpression
    expr = self.inheritCoverGrammar(self.parseBinaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1765, in parseBinaryExpression
    left = self.inheritCoverGrammar(self.parseUnaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1741, in parseUnaryExpression
    expr = self.parsePostfixExpression()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1694, in parsePostfixExpression
    expr = self.inheritCoverGrammar(self.parseLeftHandSideExpressionAllowCall)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1631, in parseLeftHandSideExpressionAllowCall
    expr = self.inheritCoverGrammar(self.parseNewExpression if self.matchKeyword('new') else self.parsePrimaryExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1525, in parsePrimaryExpression
    return self.inheritCoverGrammar(self.parseArrayInitialiser)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1239, in parseArrayInitialiser
    elements.append(self.inheritCoverGrammar(self.parseAssignmentExpression))
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1916, in parseAssignmentExpression
    expr = self.parseConditionalExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1817, in parseConditionalExpression
    expr = self.inheritCoverGrammar(self.parseBinaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1765, in parseBinaryExpression
    left = self.inheritCoverGrammar(self.parseUnaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1741, in parseUnaryExpression
    expr = self.parsePostfixExpression()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1694, in parsePostfixExpression
    expr = self.inheritCoverGrammar(self.parseLeftHandSideExpressionAllowCall)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1631, in parseLeftHandSideExpressionAllowCall
    expr = self.inheritCoverGrammar(self.parseNewExpression if self.matchKeyword('new') else self.parsePrimaryExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1528, in parsePrimaryExpression
    return self.inheritCoverGrammar(self.parseObjectInitialiser)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1399, in parseObjectInitialiser
    properties.append(self.parseObjectProperty(hasProto));
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1378, in parseObjectProperty
    value = self.inheritCoverGrammar(self.parseAssignmentExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1916, in parseAssignmentExpression
    expr = self.parseConditionalExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1817, in parseConditionalExpression
    expr = self.inheritCoverGrammar(self.parseBinaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1765, in parseBinaryExpression
    left = self.inheritCoverGrammar(self.parseUnaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1741, in parseUnaryExpression
    expr = self.parsePostfixExpression()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1694, in parsePostfixExpression
    expr = self.inheritCoverGrammar(self.parseLeftHandSideExpressionAllowCall)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1631, in parseLeftHandSideExpressionAllowCall
    expr = self.inheritCoverGrammar(self.parseNewExpression if self.matchKeyword('new') else self.parsePrimaryExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1543, in parsePrimaryExpression
    return self.parseFunctionExpression()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2780, in parseFunctionExpression
    body = self.parseFunctionSourceElements();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2627, in parseFunctionSourceElements
    body.append(self.parseStatementListItem())
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1983, in parseStatementListItem
    return self.parseStatement();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2548, in parseStatement
    return self.parseForStatement(node);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2279, in parseForStatement
    body = self.isolateCoverGrammar(self.parseStatement)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1126, in isolateCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2527, in parseStatement
    return self.parseBlock()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2005, in parseBlock
    block = self.parseStatementList()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1997, in parseStatementList
    list.append(self.parseStatementListItem())
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1983, in parseStatementListItem
    return self.parseStatement();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2552, in parseStatement
    return self.parseIfStatement(node);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2144, in parseIfStatement
    consequent = self.parseStatement();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2527, in parseStatement
    return self.parseBlock()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2005, in parseBlock
    block = self.parseStatementList()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1997, in parseStatementList
    list.append(self.parseStatementListItem())
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1983, in parseStatementListItem
    return self.parseStatement();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2568, in parseStatement
    expr = self.parseExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1949, in parseExpression
    expr = self.isolateCoverGrammar(self.parseAssignmentExpression)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1126, in isolateCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1916, in parseAssignmentExpression
    expr = self.parseConditionalExpression();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1817, in parseConditionalExpression
    expr = self.inheritCoverGrammar(self.parseBinaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1765, in parseBinaryExpression
    left = self.inheritCoverGrammar(self.parseUnaryExpression);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1741, in parseUnaryExpression
    expr = self.parsePostfixExpression()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1694, in parsePostfixExpression
    expr = self.inheritCoverGrammar(self.parseLeftHandSideExpressionAllowCall)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1141, in inheritCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1641, in parseLeftHandSideExpressionAllowCall
    args = self.parseArguments();
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1580, in parseArguments
    args.append(self.isolateCoverGrammar(self.parseAssignmentExpression))
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1126, in isolateCoverGrammar
    result = parser()
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1920, in parseAssignmentExpression
    lis = self.reinterpretAsCoverFormalsList(expr)
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1878, in reinterpretAsCoverFormalsList
    self.checkPatternParam(options, param);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 1841, in checkPatternParam
    self.validateParam(options, param, param.name);
  File "/home/antar/.miniconda3/envs/onix-env3/lib/python3.5/site-packages/js2py/translators/pyjsparser.py", line 2647, in validateParam
    elif (not options['firstRestricted']):
KeyError: 'firstRestricted'
@Slayer95
Copy link
Contributor

Slayer95 commented Aug 4, 2016 •
edited
Loading

@Antar1011, it seems the reason is the introduction of arrow functions in this file.

Is there any chance you could use PyEsprima for ES6 support or otherwise preprocess the content of the formats.js file?

I guess we could also compile an ES5-friendly formats.js with Babel and serve it in the client......

@Zarel
Copy link
Member

Zarel commented Aug 4, 2016

You could also just manually do a text replacement before passing to js2py.

@Zarel
Copy link
Member

Zarel commented Aug 4, 2016

The file shouldn't have arrow functions in the first place, Slayer95......

@Slayer95
Copy link
Contributor

Slayer95 commented Aug 4, 2016 •
edited
Loading

Introduced in 9e403b8. I wasn't aware of any such restriction anyway...

@kotarou3
Copy link
Contributor

kotarou3 commented Aug 4, 2016

Why shouldn't it have arrow functions?

@Antar1011
Copy link
Contributor Author

@Slayer95, pyesprima doesn't work with Python 3 (a Py3 version is currently in PR).

@Zarel, I'm not against pre-processing the scripts per se but this book is the extent of my JavaScript knowledge:
image

so I'll need a lot of hand-holding.

@Antar1011
Copy link
Contributor Author

Ha. Forget js2py, there's now a pip-installable pyv8:

https://blog.sqreen.io/embedding-javascript-into-python

@Antar1011 Antar1011 mentioned this issue Aug 4, 2016
Closed
@Zarel
Copy link
Member

Zarel commented Aug 4, 2016

Oh, that kind of arrow function. That's fine. I just meant onBegin, onValidateSet etc shouldn't be arrow functions because their this value is important.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Zarel @Antar1011 @kotarou3 @Slayer95