Implement infix:<ff> and family

[masak]

Eight operators in total: ff ff^ ^ff ^ff^ fff fff^ ^fff ^fff^.

I dunno why we haven't done these long ago. They are the perfect poster child for macros. As far as I can see, we can do them already. Also, I'd say they're a perfect example to have in the new examples/ directory. (See #194.)

my values = ["A", "B", "A", "B", "A"];
for values -> v {
    if v == "B" ff v == "B" {
        say(v);
    }
    else {
        say("x");
    }
}
# Result for ff: xBxBx
# Result for fff: xBABx

The reason they're perfect is that they are actually operators with internal state. If we implement this right, the operator state should be per sub clone. See these tests from the spectest suite:

# See thread "till (the flipflop operator, formerly ..)" on p6l started by Ingo
# Blechschmidt, especially Larry's reply:
# http://www.nntp.perl.org/group/perl.perl6.language/24098
# make sure calls to external sub uses the same ff each time
{
    sub check_ff($i) {
        $_ = $i;
        return (/B/ ff /D/) ?? $i !! 'x';
    }

    my $ret = "";
    $ret ~= check_ff('A');
    $ret ~= check_ff('B');
    $ret ~= check_ff('C');
    $ret ~= check_ff('D');
    $ret ~= check_ff('E');
    is $ret, 'xBCDx', 'calls from different locations use the same ff';
}

# From the same thread, making sure that clones get different states
{
    my $ret = "";
    for 0,1 {
        sub check_ff($_) { (/B/ ff /D/) ?? $_ !! 'x' }
        $ret ~= check_ff('A');
        $ret ~= check_ff('B');
        $ret ~= check_ff('C');
    }
    is $ret, 'xBCxBC', 'different clones of the sub get different ff'
}

[masak]

I think this is a correct implementation of ff:

macro infix:<ff>(lhs, rhs) {
    my active = False;
    return quasi {
        if {{{lhs}}} {
            active = True;
        }
        my result = active;
        if {{{rhs}}} {
            active = False;
        }
        result;
    };
}

Right now it fails with this error on master: No such method 'eval' for invocant of type 'Q::Block'. I think this is related to #212 and us never really supporting several statements in a quasi.

[masak]

Removed the "low-hanging-fruit" label, since this issue is blocked on another.

[masak]

Branch.

[masak]

The reason they're perfect is that they are actually operators with internal state. If we implement this right, the operator state should be per sub clone.

This won't happen on its own. Why? Because the macro will be called exactly once, at parse time, and so there will be only one "instance" of the variable active. There needs to be one per sub clone.

(See the test at the end of OP.)

I'm not clever enough today to nail down how this ought to work. But it needs to be something that re-enters the scope with the active variable whenever the sub surrounding the ff operator is cloned.

[masak]

I'm not clever enough today to nail down how this ought to work. But it needs to be something that re-enters the scope with the active variable whenever the sub surrounding the ff operator is cloned.

Thinking about this a bit more, I'm struck by the fact that the semantics we want is as if active were declared in the scope calling ff. (In the OP example, that would be the sub check_ff.)

This is exactly the mechanism proposed by S06 as my $COMPILING::new_variable;. Leaving all concerns about un-hygiene aside for the moment, I believe that would be a neater solution than forcing scopes to re-enter manually.

[masak]

Leaving all concerns about un-hygiene aside for the moment

And for when we feel up to solving the concerns about un-hygiene satisfactorily, there's a well-tested solution out there: EcmaScript 6 Symbols:

let s1 = Symbol("active");
let s2 = Symbol("active");
s1 === s2;        # false; they're distinct even with the same name
let scope = { [s1]: false, [s2]: false };
scope[s1] = true;
scope[s2];        # still false because they're distinct

This has everything we need:

• Keys/variables declared using Symbol instead of strings aren't visible to the normal user and won't collide with normal userland variables.
• Two symbols with the same name won't collide. (Meaning that two macros that accidentally grab the same name won't interact.)

I don't know the exact relation/history between (EcmaScript) symbols and (Lisp) gensyms, but I feel pretty comfortable I understand symbols. I think we want something like that when we install state-managing variables in scopes not owned by the macro itself.

[masak]

Just slapped a "currently-blocked" label on this issue. It's one of our most desirable issues to have in place (since it's at the top of the #194 list) and yet we can't move forward with it until we solve #212.

[masak]

I was toying around with having a class-based API for stateful macros. The infix:<ff> macro would come out something like this:

return class {
    property lhs;
    property rhs;

    private property active = False;

    constructor(lhs, rhs) {
        self.lhs = lhs;
        self.rhs = rhs;
    }

    eval(runtime) {
        if lhs.eval(runtime) {
            active = True;
        }
        my result = active;
        if rhs.eval(runtime) {
            active = False;
        }
        result;
    }
}

I don't know if thinking about macros as classes is fruitful in any way. To be honest I expected the class to be a better fit than the macro+quasi, but the latter is shorter and no less clear. Maybe the only interesting thing about it is that it brings us quite close to how we currently implement 007 operators with the macro nature in Perl 6. (But even that might change with #185.)

[masak]

<masak> triumphant progress report: I have macro infix:<ff> working in the `ff-macro` branch
<masak> need to do some (hopefully simple) cleaning-up, and then I can merge it to master
<masak> this has been a long time coming ;)

[masak]

I'll have to take my November me's word for it that the ff-branch used to work... it doesn't now.

$ bin/007 examples/ff.007
Variable 'v' is not declared
[...]

I haven't looked in detail, but I'm fairly certain why this happens. The two occurrences of v on this line:

if v == "B" ff v == "B" {

are going to be "dragged" into the infix:<ff> macro and processed there. The macro will spit out the generated injectile code, cocooned in a Q::Expr::BlockAdapter so that it can get the right environment. So far so hygienic.

But the two macro operands v == "B" and v == "B" don't get a similar treatment. (They should.) So currently lookup happens from the injectile's environment, which indeed does not have a v. (Nor should it.)

In other words, we should look into wrapping the unquoted things in a Q::Expr::BlockAdapter in such a way that they retain their mainline environment.

Go team hygienic macro!

[masak]

I'm not clever enough today to nail down how this ought to work.

But today, I am! 😄

The solution has been staring me in the face. I'm busy, so I'll just leave this here.

Current implementation (from examples/ff.alma):

macro infix:<ff>(lhs, rhs) is tighter(infix:<=>) {
    my active = false;
    return quasi {
        if {{{lhs}}} {
            active = true;
        }
        my result = active;
        if {{{rhs}}} {
            active = false;
        }
        result;
    };
}

Correct implementation:

macro infix:<ff>(lhs, rhs) is tighter(infix:<=>) {
    my active = new Symbol {};
    my ffFunc = new Symbol {};
    return quasi {
        let {{{active}}};
        func {{{ffFunc}}}() {
            if {{{active}}} == none {
                {{{active}}} = false;
            }
            if {{{lhs}}} {
                {{{active}}} = true;
            }
            my result = active;
            if {{{rhs}}} {
                {{{active}}} = false;
            }
            return result;
        }
        {{{ffFunc}}}();
    };
}

Re-opening so that we can fix this.

[masak]

Just dropping in here to point out a connection — previously not pointed out, I believe — between the statefulness of the ff family of macros, and the statefulness of gather from #241.