How to handle Perl-like $_ semantics in RakuAST?

[lizmat]

When I first started working on the P5 modules, the topic variable $_ was still marked as dynamic. This allowed e.g. chomp to work with Perl semantics:

multi sub chomp() { chomp CALLERS::<$_> }

In 6.d this was no longer true, which broke a lot of the P5 modules. Fortunately, jnth at the time put in a hack that allowed the P5 modules to continue to function.

However, in RakuAST we probably would not want to put in a similar hack, but instead need a more generic approach.

[lizmat]

A hack at the module level currently works in RakuAST:

use nqp;
sub foo() {
    nqp::atkey(nqp::ctxlexpad(nqp::ctxcaller(nqp::ctx)),q/$_/) = 42
}
foo;
say $_;  # 42

but fails in the legacy backend with:

Cannot modify an immutable 'Rakudo::Internals::LoweredAwayLexical' type object

Presumably this only works in RakuAST because the unreferenced $_ lexical in the outer scope is not optimized away yet like it is in the legacy backend.

So this is not a future proof solution.

[lizmat]

Another approach I was thinking about (without going full blown macro), was a trait on a subroutine that would allow changing of the AST of the callsite. Taking the chomp example, changing any call like chomp or chomp() to chomp($_).

The AST of a callsite of chomp is:

RakuAST::Statement::Expression.new(
  expression => RakuAST::Call::Name::WithoutParentheses.new(
    name => RakuAST::Name.from-identifier("chomp")
  )
)

This would need to be changed to:

RakuAST::Statement::Expression.new(
  expression => RakuAST::Call::Name::WithoutParentheses.new(
    name => RakuAST::Name.from-identifier("chomp"),
    args => RakuAST::ArgList.new(
      RakuAST::Var::Lexical.new("\$_")
    )
  )
)

In fact as simple as adding the args argument. Which actually we have an API for already in RakuAST::Call.replace-args.

So an idea for an interface would be:

sub chomp() will ast(&add-topic) { ... }

and the trait_mod being something like:

sub trait_mod:<will>(&AST, &modify, :$ast!) {
     modify($AST)
}

and add-topic being something like:

sub add-topic($AST) {
    $AST.replace-args(
      RakuAST::ArgList.new(RakuAST::Var::Lexical.new("\$_"))
    )
}

Now, all is in the timing as always. Traits are currently called on the creation of the sub: we want it to be called for every call-site. So maybe a new trait-type would be needed. Or maybe the trait should just set a flag somewhere that would trigger calling modify from the above example during the CHECK phase.

Anyways, I think all the movable parts are already available, we just need to plug it in at the right place.

[lizmat]

Alternately, we could just make the implicit use of a caller's $_ a runtime error in the P5 modules.

[FCO]

Another approach I was thinking about (without going full blown macro), was a trait on a subroutine that would allow changing of the AST of the callsite. Taking the chomp example, changing any call like chomp or chomp() to chomp($_).

The AST of a callsite of chomp is:

RakuAST::Statement::Expression.new(
  expression => RakuAST::Call::Name::WithoutParentheses.new(
    name => RakuAST::Name.from-identifier("chomp")
  )
)

This would need to be changed to:

RakuAST::Statement::Expression.new(
  expression => RakuAST::Call::Name::WithoutParentheses.new(
    name => RakuAST::Name.from-identifier("chomp"),
    args => RakuAST::ArgList.new(
      RakuAST::Var::Lexical.new("\$_")
    )
  )
)

In fact as simple as adding the args argument. Which actually we have an API for already in RakuAST::Call.replace-args.

So an idea for an interface would be:

sub chomp() will ast(&add-topic) { ... }

and the trait_mod being something like:

sub trait_mod:<will>(&AST, &modify, :$ast!) {
     modify($AST)
}

and add-topic being something like:

sub add-topic($AST) {
    $AST.replace-args(
      RakuAST::ArgList.new(RakuAST::Var::Lexical.new("\$_"))
    )
}

Now, all is in the timing as always. Traits are currently called on the creation of the sub: we want it to be called for every call-site. So maybe a new trait-type would be needed. Or maybe the trait should just set a flag somewhere that would trigger calling modify from the above example during the CHECK phase.

Anyways, I think all the movable parts are already available, we just need to plug it in at the right place.

This is similar to what I want to use to translate map blocks to sql at compile-time on Red. I was originally thinking of using ASTQuery for that, but if that trait exists, it seems it would be even easier to use that.

[FCO]

Just explaining a bit more: the plan is to get the original arg (the block), translate that into Red::AST, and add that as a named parameter. But that mean the trait should also work for methods. Would it work for methods? (I suppose not)