Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Python3 grammar for Python3 target #572

Merged
merged 1 commit into from
Sep 17, 2017
Merged

Conversation

aptrishu
Copy link
Contributor

Please suggest a better name of the python target grammar file. Also, the documentation needs improvement I think.

@parrt
Copy link
Member

parrt commented Mar 12, 2017

Thanks!

@parrt
Copy link
Member

parrt commented Mar 12, 2017

Oops. causes a build failure:

[ERROR] /home/travis/build/antlr/grammars-v4/Python3_py.g4 [31:8]: grammar name Python3 and file name Python3_py.g4 differ

@aptrishu
Copy link
Contributor Author

Yes so, should I put same name for both files ? or change the grammar name ?

@parrt
Copy link
Member

parrt commented Mar 12, 2017

Diff files have to have diff grammar names inside. Make new file have grammar Python3_py; inside I guess.

@aptrishu
Copy link
Contributor Author

@parrt
Thanks, I updated the grammar name.
waiting for the checks to pass.

@parrt
Copy link
Member

parrt commented Mar 12, 2017

Hmm...we apparently need to modify the pom.xml for that so it doesn't try to build with java target. @teverett any ideas?

@teverett
Copy link
Member

Can this pull wait until after antlr4-4.7, so that we can make fixed once, not twice?

@aptrishu
Copy link
Contributor Author

Also, I have modified str to strr in grammar to avoid conflict. str is a method in Python3.
Does that need a mention in the grammar or docs somewhere ?

@parrt
Copy link
Member

parrt commented Mar 12, 2017

nope.

@aptrishu
Copy link
Contributor Author

When is the 4.7 release scheduled? It'd also be better we make a doc for such grammars which can vary target to target. Though I have no idea how many grammars in this repo are target dependent.

@teverett
Copy link
Member

teverett commented Mar 12, 2017 via email

@KvanTTT
Copy link
Member

KvanTTT commented Mar 13, 2017

Can we organize grammars for different runtimes by the following way?

RuntimeName1/GrammarName.g4
...
RuntimeNameN/GrammarName.g4

RuntimeName can be one of the following items:

  • Java
  • C# (or CSharp)
  • C#-Sharwell
  • Python2 (or just Python for both versions)
  • Python3
  • JavaScript
  • C++ (or CPlusPlus)
  • Go
  • Swift

But GrammarName should be the same for one grammar.

Such structure unify grammar processing by different projects. See PHP grammar for example.

@parrt
Copy link
Member

parrt commented Mar 13, 2017

Well good idea but I'm hoping to keep grammars action free. Python is special given it's INDENT/DEDENT rules.

@aptrishu
Copy link
Contributor Author

I'm trying to make Python3 grammar action free. I'll replace INDENT/DEDENTs with common delimiters like '{'.

@parrt
Copy link
Member

parrt commented Mar 14, 2017

I'd go with INDENT/DEDENT to avoid confusion and "this isn't python" issues from people ;)

@aptrishu
Copy link
Contributor Author

aptrishu commented Mar 14, 2017

Yes true.That'd create confusion. Using INDENT/DEDENT is better.
Thanks. :)

@KvanTTT
Copy link
Member

KvanTTT commented Mar 14, 2017

@parrt it's not possible to keep grammars action free. Context-sensitivity rules require grammar actions.

@parrt
Copy link
Member

parrt commented Mar 14, 2017

Very few grammars require semantic predicates.

@teverett
Copy link
Member

teverett commented Apr 2, 2017

@aptrishu this pull request has failed the travis build with

[INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[3,1] class, interface, or enum expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[3,37] '.' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[4,5] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[4,40] '.' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[5,7] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[129,15] <identifier> expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[129,18] illegal start of expression [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[129,19] illegal start of type [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[130,19] illegal start of expression [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[130,20] illegal start of type [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[132,18] <identifier> expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[132,25] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[134,27] <identifier> expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[134,30] <identifier> expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[134,31] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[166,39] <identifier> expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[166,40] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[229,54] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[230,64] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[231,62] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[232,49] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,27] '(' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,43] ')' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,44] not a statement [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,46] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,52] not a statement [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,59] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,67] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[234,83] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[235,35] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[236,25] 'else' without 'if' [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[236,29] illegal start of expression [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[237,33] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[237,84] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[238,70] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,55] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,58] '(' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,71] ')' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,72] 'else' without 'if' [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,77] not a statement [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,78] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,31] '(' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,50] ')' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,51] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[242,37] illegal start of expression [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[242,42] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[243,40] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[244,60] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[245,95] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,29] 'else' without 'if' [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,33] illegal start of expression [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,34] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[247,55] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[247,72] not a statement [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[247,80] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[248,72] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[249,55] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[257,41] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[264,41] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[271,41] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[278,41] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[285,41] ';' expected [ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[292,41] ';' expected [INFO] 63 errors [INFO] -----------

@aptrishu
Copy link
Contributor Author

aptrishu commented Apr 2, 2017

Looks weird, I'll look into it tomorrow.

@aptrishu
Copy link
Contributor Author

aptrishu commented Apr 3, 2017

@teverett can I see on which test it failed/produced such output?

@teverett
Copy link
Member

teverett commented Apr 3, 2017

Best way is for you to click the "details" button which takes you to Travis and you can see the logs

@aptrishu
Copy link
Contributor Author

aptrishu commented Apr 3, 2017

Yeah, log is quite big(Also it only shows the tests being executed,their pass/failure) that's why I asked for an alternative where I could see on which specific file it produced such output. Ok I'll look into the logs. Thanks.

@parrt
Copy link
Member

parrt commented Sep 16, 2017

@teverett are we ok to merge this?

@teverett
Copy link
Member

@parrt well, no, because the tests don't pass.

@teverett
Copy link
Member

Here are the specific log messages for the failure

[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[3,1] class, interface, or enum expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[3,37] '.' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[4,5] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[4,40] '.' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[5,7] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[133,15]  expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[133,18] illegal start of expression
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[133,19] illegal start of type
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[134,19] illegal start of expression
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[134,20] illegal start of type
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[136,18]  expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[136,25] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[138,27]  expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[138,30]  expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[138,31] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[170,39]  expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[170,40] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[236,54] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[237,64] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[238,62] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[239,49] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,27] '(' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,43] ')' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,51] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,47] variable declaration not allowed here
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,67] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[241,83] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[242,35] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[243,29] illegal start of expression
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[243,25] 'else' without 'if'
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[244,33] illegal start of expression
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[244,43] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[244,83] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[245,70] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,55] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,58] '(' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,71] ')' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,77] not a statement
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,78] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[246,72] 'else' without 'if'
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[248,39] not a statement
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[248,50] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[249,44] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[250,40] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[251,60] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[252,95] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[253,33] illegal start of expression
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[253,29] 'else' without 'if'
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[254,55] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[254,72] not a statement
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[254,80] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[255,72] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[256,55] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[264,41] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[271,41] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[278,41] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[285,41] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[292,41] ';' expected
[ERROR] /home/travis/build/antlr/grammars-v4/python3/target/generated-sources/antlr4/Python3_pyLexer.java:[299,41] ';' expected
[INFO] 59 errors 

@teverett
Copy link
Member

ok the problem is trivial: pthyon3_p4.g4 contains python-specific code. With PR #883 I've pulled the code from @aptrishu, merged trunk, and then moved the python-specific .g4 into it's own dir. If we are ok with this approach i'll pull PR #883 and close PR #572.

@parrt
Copy link
Member

parrt commented Sep 16, 2017 via email

@teverett teverett mentioned this pull request Sep 17, 2017
@teverett teverett merged commit 2fc0e30 into antlr:master Sep 17, 2017
@KvanTTT KvanTTT added python2 and removed python2 labels Nov 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants