Document /<foo ...>/ syntax somewhere

[zoffixznet]

It's a special syntax that lets users write stuff like their own versions of stuff like <before ...>. (description copied from rakudo/rakudo#1634 (comment))

[…] it is the syntax that lets us parse <before abc> and similar. Unlike with <foo: ...> and <foo(...)>, where the args are parsed by the MAIN language, the <foo ...> form parses the rest of the assertion using the regex language, wraps it in a closure, and passes it to foo. Thus it's a general syntax for higher-order regex operators - which is just what before/after are.

I guess maybe somewhere in the "advanced" section or grammars is a good place to put this. Just to separate it from all the common syntax novice users might be reading through.

[JJ]

Grammars or regexes?

[zoffixznet]

🤷‍♀️

[mryan]

OK. Had a look at this on and off during the day. A few preliminary points (as I see it - correct where necessary):

• Using this feature, , requires the user to define a method in the grammar that will receive a compiled regex as it's only argument.
• You can't do anything with that regex other than run it "against" a cursor object
• A method in a grammar must return a cursor to its caller
• Looking at all the existing documentation, I can't see a case where a method is declared in a grammar which (necessarily) returns a cursor with one exception here: https://docs.perl6.org/language/grammars#Methods_in_Grammar
  That example is a very specific case in that the method is TOP and hence it's called first thing which means that any regex/token/rule in the grammar can be returned as the "cursor that must be returned" because the "slate is clean" so to speak. [I wish I could explain this better but I cant :-)]
• Looking at the cases where this feature is used in the existing language ( <before ...> and <after ...> , you can see the call to '!cursor_start_cur' and the work to populate the new cursor before running the regex against it passing it as a return value - what I'm getting at is I can't see a way to make this available to users without dragging them into the a lot of detail about cursor creation and management.
• despite that, I did make some head way with creating an example, <SOLscan ...> (Start-of-line scan) which will look at the current line being passed for the regex supplied but the use case is complicated and contrived and requires explaining a fair bit of the internal regex-parsing/cursor details.

So, before I go further, I need to know if

• there's a whole heap of stuff I'm not getting; and

• if I'm reasonably on track, do we want to expose some of the cursor management details to end users so they can build there own meta parsing rules with this feature; or

• "take away" this feature. i.e. make it an "internals only" type thing; or

• leave things as is - its exposed for you to use if you can, but there's no documentation on it.

None of these options look all that good to me. I need some advice.
Lastly, either this isn't really a "good first issue" or I've got a lot to learn...

[JJ]

You look reasonably on track, but you probably need to show your examples to see how it goes. You can't take away this feature; if it's internal, it should not be documented, but if it's not...

[JJ]

Gist documented what's been done so far here: https://gist.github.com/mryan/dc8c890de862876abc4f87f55d974070
Thanks @mryan

[AlexDaniel]

This looks extremely useful, I'm +1 for adding it. Generally, we should first look if there are any tests in roast for it. If there are, then we document it right away. If not, then we don't…

If there are no tests for it right now, maybe add some? I'm not sure which part of it exactly is implementation-specific…

[mryan]

I've posted a question to stack overflow asking how to use this feature. Need to see how it's used before being able to document how it's used.
https://stackoverflow.com/questions/50210456/how-can-i-use-the-perl6-regex-metasyntax-foo-regex

[JJ]

Check out @TimToady 's comment in the IRC https://colabti.org/irclogger/irclogger_log/perl6?date=2018-05-08#l150