Grammar breaks intermittently after changing role to grammar

[hythm7]

I have a grammar that broke after updating rakudo to latest: i made a small version of the grammar in this gist : https://gist.github.com/hythm7/dedb6638105029f7b103b7f99f04e323

in the gist there is output file, with the perl6 version that works consistently. basically latest breaks intermittently

Also please see https://colabti.org/irclogger/irclogger_log/perl6?date=2019-05-28#l723

Please check the comments below, I reopened the issue again because the grammar now breaks if used as grammar instead of a role.

Environment

• Operating system: Linux
• Compiler version (perl6 -v): This is Rakudo version 2019.03.1 built on MoarVM version 2019.03
  implementing Perl 6.d.

[hythm7]

Closing since building rakudo with rakudebrew build moar master instead of rakudobrew build moar fixed the issue.

[hythm7]

Reopenning:
If changed role to grammar I see the issue again:

#!/usr/bin/env perl6

grammar Pkg {         # no issue if useing role

  token TOP { <pkg> }

  proto token pkg { * }

  token pkg:sym<tag>     { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <build> <.hyphen> <tag> <ext>? }
  token pkg:sym<build>   { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <form> <ext>? }
  token pkg:sym<arch>    { <name> <.hyphen> <version> <.hyphen> <arch> }
  token pkg:sym<version> { <name> <.hyphen> <version> }
  token pkg:sym<name>    { <name> }

  token name { [ [ <.alnum>+ <!before <dot>> ]+ % <hyphen> ] }
  token version  { [ [ <.digit>+ | '*' ]+ % <dot> ] <plus>? }
  token arch { 'x86_64' | 'i386' }
  token build { <.digit>+ }
  token tag  { <.alnum>+ }

  token ext { <.dot> 'ext' }

  token dot    { '.' }
  token plus   { '+' }
  token hyphen { '-' }
}

grammar Cmd {
  also is Pkg;

  rule TOP    { <pkgs> }

  token pkgs { [ <.pkg>+ % <.blank> ] }

}

say so Cmd.parse('raku-0.0.1');

Output:

hythm@galaxy$ for i in $(seq 10); do /tmp/grammar.p6; done
False
False
True
True
False
True
False
True
False
False

hythm@galaxy$ perl6 --version
This is Rakudo version 2019.03.1-572-g89f8f60 built on MoarVM version 2019.05-36-ge865f43
implementing Perl 6.d.

[hythm7]

I tried the above grammar on few releases of moar, below table shows that grammar is working (consistently) with 2017.07, after that it breaks (intermittently) due to two reasons:

declaring grammar "Pkg" with grammar vs with role

Release  grammar   role
2017.07     ✓       ✓
2018.04     ✓       ✓
2018.05     ✗       ✗ 
2018.12     ✗       ✗ 
master      ✗       ✓

[ugexe]

Your example doesn't help -- I changed your code grammar to role and got the same behavior

[hythm7]

Are you running latest perl6?

#!/usr/bin/env perl6

role Pkg {

  token TOP { <pkg> }

  proto token pkg { * }

  token pkg:sym<tag>     { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <build> <.hyphen> <tag> <ext>? }
  token pkg:sym<build>   { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <form> <ext>? }
  token pkg:sym<arch>    { <name> <.hyphen> <version> <.hyphen> <arch> }
  token pkg:sym<version> { <name> <.hyphen> <version> }
  token pkg:sym<name>    { <name> }

  token name { [ [ <.alnum>+ <!before <dot>> ]+ % <hyphen> ] }
  token version  { [ [ <.digit>+ | '*' ]+ % <dot> ] <plus>? }
  token arch { 'x86_64' | 'i386' }
  token build { <.digit>+ }
  token tag  { <.alnum>+ }

  token ext { <.dot> 'ext' }

  token dot    { '.' }
  token plus   { '+' }
  token hyphen { '-' }
}

grammar Cmd {
  also does Pkg;

  rule TOP    { <pkgs> }

  token pkgs { [ <.pkg>+ % <.blank> ] }

}

say so Cmd.parse('raku-0.0.1');


hythm@galaxy$ for i in $(seq 7); do /tmp/grammar.p6; done
True
True
True
True
True
True
True

hythm@galaxy$ perl6 --version
This is Rakudo version 2019.03.1-594-gbed0695 built on MoarVM version 2019.05-53-gcce71cd
implementing Perl 6.d.

[hythm7]

/usr/bin/env perl6

grammar Pkg {

  token TOP { <pkg> }

  proto token pkg { * }

  token pkg:sym<tag>     { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <build> <.hyphen> <tag> <ext>? }
  token pkg:sym<build>   { <name> <.hyphen> <version> <.hyphen> <arch> <.hyphen> <form> <ext>? }
  token pkg:sym<arch>    { <name> <.hyphen> <version> <.hyphen> <arch> }
  token pkg:sym<version> { <name> <.hyphen> <version> }
  token pkg:sym<name>    { <name> }

  token name { [ [ <.alnum>+ <!before <dot>> ]+ % <hyphen> ] }
  token version  { [ [ <.digit>+ | '*' ]+ % <dot> ] <plus>? }
  token arch { 'x86_64' | 'i386' }
  token build { <.digit>+ }
  token tag  { <.alnum>+ }

  token ext { <.dot> 'ext' }

  token dot    { '.' }
  token plus   { '+' }
  token hyphen { '-' }
}

grammar Cmd {
  also is Pkg;

  rule TOP    { <pkgs> }

  token pkgs { [ <.pkg>+ % <.blank> ] }

}

say so Cmd.parse('raku-0.0.1');


hythm@galaxy$ for i in $(seq 7); do /tmp/grammar.p6; done
True
True
False
True
False
True
False

hythm@galaxy$ perl6 --version
This is Rakudo version 2019.03.1-594-gbed0695 built on MoarVM version 2019.05-53-gcce71cd
implementing Perl 6.d.