Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perl 6 parser bugging out on a comment thinking it is a real var #6691

Open
p6rt opened this issue Jan 23, 2019 · 16 comments
Open

perl 6 parser bugging out on a comment thinking it is a real var #6691

p6rt opened this issue Jan 23, 2019 · 16 comments
Labels

Comments

@p6rt
Copy link

@p6rt p6rt commented Jan 23, 2019

Migrated from rt.perl.org#133791 (status was 'open')

Searchable as RT133791$

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 23, 2019

From warren.music@gmail.com

Hello​:

I ran into this while setting up a post test for json
in bailador. While compiling it flags the commented
line at the end as bad when the fail should be on the
check of request.body[0].

It happened with the latest rakudo built from scratch
as of Jan 23rd 2019 as well as rakudo-star 2018.10.

Linux Mint system, 64 bit.

# --->perl6 t1.pl6
  # ===SORRY!=== Error while compiling
/home/userx/p6d/tests/latester/t1.pl6
  # Variable '%bb' is not declared
  # at /home/userx/p6d/tests/latester/t1.pl6​:97
  # ------> #pukes here #say %bb{"name"};

  # code snippet that causes the parser to think
  # the commented code below is not commented
  if request.body[0] == "{" { say "JSON"} else {say "NOTJSON"};

  # #my %bb = from-json(request.body);
  #
  # this one pukes
  #pukes here #say %bb{"name"};

Loading

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 25, 2019

From @AlexDaniel

Usually this happens when you have an unclosed string somewhere earlier in your code.

That is​:

say "foo; ← oops! Forgot the closing "

# $a ← we think that this is a comment, but actually it's part of the string above!
On 2019-01-23 01​:27​:08, warren.music@​gmail.com wrote​:

Hello​:

I ran into this while setting up a post test for json
in bailador. While compiling it flags the commented
line at the end as bad when the fail should be on the
check of request.body[0].

It happened with the latest rakudo built from scratch
as of Jan 23rd 2019 as well as rakudo-star 2018.10.

Linux Mint system, 64 bit.

# --->perl6 t1.pl6
# ===SORRY!=== Error while compiling
/home/userx/p6d/tests/latester/t1.pl6
# Variable '%bb' is not declared
# at /home/userx/p6d/tests/latester/t1.pl6​:97
# ------> #pukes here #say %bb{"name"};

# code snippet that causes the parser to think
# the commented code below is not commented
if request.body[0] == "{" { say "JSON"} else {say "NOTJSON"};

# #my %bb = from-json(request.body);
#
# this one pukes
#pukes here #say %bb{"name"};

Loading

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 25, 2019

The RT System itself - Status changed from 'new' to 'open'

Loading

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 26, 2019

From @timo

I believe the problem comes from `"{"` which actually starts an
interpolated code block containing a string immediately. That's also why
it doesn't complain about the "else" being in an odd place; it's also
inside the string!

So here's an equivalent piece of code that shows what's wrong​:

if request.body[0] == "" ~ do { qq⟨{ say "JSON"} else {say "NOTJSON"};
# my %bb = 1234 => 99;
(and here comes the closing quote for the qq that was missing in the original code​: ⟩

Simplifying a tiny bit more​:

if request.body[0] == "" ~ (say "JSON") ~ " else " ~ (say "NOTJSON") ~ ";
# my %bb = 1234 => 99;
again no closing double-quotes

Does that help?

On 25/01/2019 07​:20, Aleks-Daniel Jakimenko-Aleksejev via RT wrote​:

Usually this happens when you have an unclosed string somewhere earlier in your
code.

That is​:

say "foo; ← oops! Forgot the closing "

# $a ← we think that this is a comment, but actually it's part of the string
above!
On 2019-01-23 01​:27​:08, warren.music@​gmail.com wrote​:

Hello​:

I ran into this while setting up a post test for json
in bailador. While compiling it flags the commented
line at the end as bad when the fail should be on the
check of request.body[0].

It happened with the latest rakudo built from scratch
as of Jan 23rd 2019 as well as rakudo-star 2018.10.

Linux Mint system, 64 bit.

# --->perl6 t1.pl6
# ===SORRY!=== Error while compiling
/home/userx/p6d/tests/latester/t1.pl6
# Variable '%bb' is not declared
# at /home/userx/p6d/tests/latester/t1.pl6​:97
# ------> #pukes here #say %bb{"name"};

# code snippet that causes the parser to think
# the commented code below is not commented
if request.body[0] == "{" { say "JSON"} else {say "NOTJSON"};

# #my %bb = from-json(request.body);
#
# this one pukes
#pukes here #say %bb{"name"};

Loading

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 28, 2019

From warren.music@gmail.com

Oh, for me I just moved on to a proper bit of code.
I filed the bug for the perfection of the parser so
it's less exploitable. :)

"I code therefore I am!"

On 1/26/19 8​:17 PM, Timo Paulssen via RT wrote​:

I believe the problem comes from `"{"` which actually starts an
interpolated code block containing a string immediately. That's also why
it doesn't complain about the "else" being in an odd place; it's also
inside the string!

So here's an equivalent piece of code that shows what's wrong​:

if request.body[0] == "" ~ do { qq⟨{ say "JSON"} else {say "NOTJSON"};
# my %bb = 1234 => 99;
(and here comes the closing quote for the qq that was missing in the original code​: ⟩

Simplifying a tiny bit more​:

if request.body[0] == "" ~ (say "JSON") ~ " else " ~ (say "NOTJSON") ~ ";
# my %bb = 1234 => 99;
again no closing double-quotes

Does that help?

On 25/01/2019 07​:20, Aleks-Daniel Jakimenko-Aleksejev via RT wrote​:

Usually this happens when you have an unclosed string somewhere earlier in your
code.

That is​:

say "foo; ← oops! Forgot the closing "

# $a ← we think that this is a comment, but actually it's part of the string
above!
On 2019-01-23 01​:27​:08, warren.music@​gmail.com wrote​:

Hello​:

I ran into this while setting up a post test for json
in bailador. While compiling it flags the commented
line at the end as bad when the fail should be on the
check of request.body[0].

It happened with the latest rakudo built from scratch
as of Jan 23rd 2019 as well as rakudo-star 2018.10.

Linux Mint system, 64 bit.

# --->perl6 t1.pl6
# ===SORRY!=== Error while compiling
/home/userx/p6d/tests/latester/t1.pl6
# Variable '%bb' is not declared
# at /home/userx/p6d/tests/latester/t1.pl6​:97
# ------> #pukes here #say %bb{"name"};

# code snippet that causes the parser to think
# the commented code below is not commented
if request.body[0] == "{" { say "JSON"} else {say "NOTJSON"};

# #my %bb = from-json(request.body);
#
# this one pukes
#pukes here #say %bb{"name"};

Loading

@p6rt
Copy link
Author

@p6rt p6rt commented Jan 28, 2019

From warren.music@gmail.com

A missing string never got flagged though. When I comment out the
line I call the C line everything worked.

I submitted this as a bug since being the worse kind of exploiter
I love seeing things like this. I'd go gung ho at using
this for malicious activities if I was a hacker.

"Concentrationism"

On 1/25/19 1​:20 PM, Aleks-Daniel Jakimenko-Aleksejev via RT wrote​:

Usually this happens when you have an unclosed string somewhere earlier in your
code.

That is​:

say "foo; ← oops! Forgot the closing "

# $a ← we think that this is a comment, but actually it's part of the string
above!
On 2019-01-23 01​:27​:08, warren.music@​gmail.com wrote​:

Hello​:

I ran into this while setting up a post test for json
in bailador. While compiling it flags the commented
line at the end as bad when the fail should be on the
check of request.body[0].

It happened with the latest rakudo built from scratch
as of Jan 23rd 2019 as well as rakudo-star 2018.10.

Linux Mint system, 64 bit.

# --->perl6 t1.pl6
# ===SORRY!=== Error while compiling
/home/userx/p6d/tests/latester/t1.pl6
# Variable '%bb' is not declared
# at /home/userx/p6d/tests/latester/t1.pl6​:97
# ------> #pukes here #say %bb{"name"};

# code snippet that causes the parser to think
# the commented code below is not commented
if request.body[0] == "{" { say "JSON"} else {say "NOTJSON"};

# #my %bb = from-json(request.body);
#
# this one pukes
#pukes here #say %bb{"name"};

Loading

@JJ
Copy link

@JJ JJ commented May 7, 2020

Not sure this is an actual issue? It seems to work as expected, right?

Loading

@AlexDaniel AlexDaniel added the LTA label May 7, 2020
@AlexDaniel
Copy link
Contributor

@AlexDaniel AlexDaniel commented May 7, 2020

@JJ not really, it is still an issue because rakudo doesn't indicate that it is parsing a string. Basically: say “foo ; # $bar. User forgot to close the string and the parser tripped over something that looks like a variable.

Loading

@JJ
Copy link

@JJ JJ commented May 30, 2020

Again, this seems to be an issue of ordering. It's interpreting the string before checking out if it's finished or not. It's got nothing to do with it having a semicolon or a comment.

say “foo $bar

This does exactly the same. And I think this is simply a call. Even if the quotes are closed, this is going to fail, so it probably makes sense to fail as soon as possible"

Loading

@AlexDaniel
Copy link
Contributor

@AlexDaniel AlexDaniel commented May 30, 2020

@JJ not really, it is still an issue because rakudo doesn't indicate that it is parsing a string. Basically: say “foo ; # $bar. User forgot to close the string and the parser tripped over something that looks like a variable.

Loading

@JJ
Copy link

@JJ JJ commented May 30, 2020

@AlexDaniel it's got nothing to do with ; or #.

say "$bar

also fails. Again, it's a matter of interpretation, and it seems to me it's a judgement call. The compiler does not care if the user failed to close the string or not. Even if that was the case, $bar is going to fail because it's not defined. So, judgement call, and an example of failing as soon as possible. is there any directive anywhere that says that some syntax errors should have precedence over others?

Loading

@AlexDaniel
Copy link
Contributor

@AlexDaniel AlexDaniel commented May 30, 2020

@JJ I don't know how else I can explain this. The main issue here is that otherwise valid code results in very confusing (LTA) error messages when the user forgets to close their strings.

For example:

say “foo ; my $x

Trips on $x saying it should be declared first, even though the code is attempting to do just that.

Or:

say “foo ; # $x

The variable here is part of the comment, so it should be ignored, yet the compiler has no idea because it is still parsing a string.

Yeah, of course, there is a syntax error, no doubt there. But the compiler is producing an error message much, much later, possibly a few lines below, and it does not indicate in any way that it was parsing a string when it happened.

Loading

@JJ
Copy link

@JJ JJ commented May 30, 2020

@AlexDaniel the obverse is also true. You're saying that when there's a semicolon in a string, the compiler should somehow infer that maybe the intention was to close quotes before that. That's simply wrong, and could lead to all kind of problems. In general, interpreting things inside a string as intentions to do anything at all is a slippery slope that will cause many problems (and in any case general slowness).
You say that

say “foo ; # $x

should be interpreted as an unfinished string (and right before the colon). But what about this?

say "foo #my $x"

Should we interpret here that there's also a semicolon missing? In general, should we test-compile every string looking for possible syntax errors? In the example, there's a CR in the middle; but CRs are also valid part of strings. Why should there be a difference? Why does the presence of a $ magically turns a string into something that, maybe, is an example of code beyond a non-closed quote? What about this one

say "this; or maybe $that; this is a hash -> #;

Where should the compiler close the quote? I mean, clearly therein lies dragons.

Let's assume that what is within a string is intended to be within a string, and let's just fail if there's something inside it that's a possible syntax error, like a variable. It's a judgment call, and it's the right one far as I'm concerned.

I agree that the message is LTA. But an error is something the compiler has not understood real well. Ascertaining the intention of the user would mean using certain heuristics (there's one or several CRs; there's one or several ; or maybe }) which would make the error message marginally better and would certainly make it heavier to process after the error, backtracking to try and find stuff...

So, again, probably we should close this.

Loading

@AlexDaniel
Copy link
Contributor

@AlexDaniel AlexDaniel commented May 30, 2020

I agree that the message is LTA

Yes. This ticket is marked as LTA.

So, again, probably we should close this.

What?

The error message should indicate whether the compiler was parsing a string when the compile-time error happened. As simple as that.

Loading

@JJ
Copy link

@JJ JJ commented May 30, 2020

I agree that the message is LTA

Yes. This ticket is marked as LTA.

So, again, probably we should close this.

What?

The error message should indicate whether the compiler was parsing a string when the compile-time error happened. As simple as that.

That makes sense...

Loading

@JJ
Copy link

@JJ JJ commented May 16, 2021

This seems to be a LHF: it needs to change and message and/or fail earlier. Anyone?

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants