Gather issues

[dumarchie]

The Problem

A Seq produced by gather does not declare itself to be lazy. Invoking .gist on an infinite gather makes a subsequent .is-lazy hang.

Expected Behavior

The following tests:

use Test;
plan 3;

my $seq = gather {
    my $i = 0;
    while 1 { take $i++ }
};

ok $seq.is-lazy, '.is-lazy returns True on infinite gather';

is $seq.gist, (0..100).list.gist, '.gist';

my $test = start { $seq.is-lazy };
await Promise.anyof($test, Promise.in(5));
ok $test, '.is-lazy does not hang after .gist';

Should output

1..3
ok 1 - .is-lazy returns True on infinite gather
ok 2 - .gist
ok 3 - .is-lazy does not hang after .gist

Actual Behavior

The actual test output is

1..3
not ok 1 - .is-lazy returns True on infinite gather
# Failed test '.is-lazy returns True on infinite gather'
# at gather.is-lazy.pl6 line 9
ok 2 - .gist
not ok 3 - .is-lazy does not hang after .gist
# Failed test '.is-lazy does not hang after .gist'
# at gather.is-lazy.pl6 line 15
# Looks like you failed 2 tests of 3

Environment

• Operating system: Windows 10
• Compiler version (perl6 -v):
  This is Rakudo Star version 2019.03.1 built on MoarVM version 2019.03
  implementing Perl 6.d.

[lizmat]

An easy fix would be to just mark the Rakudo::Iterator::Gather iterator as lazy. However, that breaks installation with:

===SORRY!===
Cannot sort a lazy list

So that's not an option. So I guess we need a syntactic way to mark a gather block as lazy, so it can be passed to the iterator creation.

In preparation for that, I have added an "is-lazy" attribute to the underlying iterator in b6afa71

[lizmat]

After 07072f3 the following test runs to completion:

use Test;
plan 3;

my $seq = gather {
    my $i = 0;
    while 1 { take $i++ }
} :is-lazy;

ok $seq.is-lazy, '.is-lazy returns True on infinite gather';

is $seq.gist, '(...)', '.gist';

my $test = start { $seq.is-lazy };
await Promise.anyof($test, Promise.in(5));
ok $test, '.is-lazy does not hang after .gist';

Pinging @jnthn to see if he's ok with this change.

[jnthn]

So I guess we need a syntactic way to mark a gather block as lazy

We do; it's spelled lazy gather.

[jnthn]

Pinging @jnthn to see if he's ok with this change.

No, not when there's already a fine way that's shorter and more regular. :-)

Ah, and to add: that gather is not lazy by default, and requires such marking, is by design.

[lizmat]

Ok, I'll revert the last one. And look at making lazy smarter, so it can tell the gather iterator to be lazy, rather than embedding it in another iterator marking itself lazy.

[jnthn]

Be very careful; it must only do that optimization if it is immediately receiving the gather block.

[lizmat]

Yeah, that is becoming difficult. The whole Lazy iterator is based on the fact that it will not call the source.iterator until the first pull-one is done. So essentially we would need to know whether an iterator will be lazy before it gets instantiated. Chicken and egg problem. So closing this ticket now.

[dumarchie]

@lizmat, I accept that the first is-lazy returns false by design. But still the second is-lazy should not hang and this ticket should not be closed until that is fixed or documented.

Note that I used .gist to trigger the hang because that's what the REPL calls on the result of most statements. The hang is actually caused by the underlying .cache call, so I reduced the test to

use Test;
plan 2;

my $seq = gather {
    my $i = 0;
    while 1 { take $i++ }
};
ok !$seq.is-lazy, '.is-lazy on infinite gather';

$seq.cache;

my $test = start { $seq.is-lazy };
await Promise.anyof($test, Promise.in(5));
ok $test, '.is-lazy does not hang after .cache';

and on the latest Rakudo build the output is:

1..2
ok 1 - .is-lazy on infinite gather
not ok 2 - .is-lazy does not hang after .cache
# Failed test '.is-lazy does not hang after .cache'
# at gather.is-lazy.pl6 line 14
# You failed 1 test of 2

[lizmat]

This behaviour is caused by List.is-lazy doing a $!todo.reify-until-lazy, because after the $seq.cache call, this is essentially lazy List semantics at work.

FWIW, I've never fully understood why the caching semantics of Seq would need to use the List semantics. Perhaps @jnthn can enlighten.

[lizmat]

I'm closing this issue now. Please re-open if you disagree.