LazyCollection chunk using generator delegation

[ankr]

• Laravel Version: 6.18.0
• PHP Version: 7.3.13

Description:

When using generator delegation in combination with LazyCollection::chunk() and a chunk size greater than amount of values yielded from initial generator, all values from delegated generator are lost.

Using a chunk smaller than (or equal to) the amount of values yielded from initial generator - everything works as expected.

Steps To Reproduce:

Make generator

<?php
function gen($stop = false) {
    yield 1;
    yield 2;
    yield 3;

    if (! $stop) {
        yield from gen(true);
    }
}

Make collection

<?php
$collection = LazyCollection::make(function () {
    yield from gen();
});

Chunk and inspect collection

<?php
// all good
$collection->chunk(1)->toArray(); // [[1], [2], [3], [1], [2], [3]]
$collection->chunk(2)->toArray(); // [[1, 2], [3, 1], [2, 3]]
$collection->chunk(3)->toArray(); // [[1, 2, 3], [1, 2, 3]]

// starting to loose values
$collection->chunk(4)->toArray(); // [[1, 2, 3]]
$collection->chunk(5)->toArray(); // [[1, 2, 3]]

[driesvints]

@JosephSilber I guess this isn't possible with LazyCollections?

[JosephSilber]

This has nothing to do with lazy collections. You can see the same behavior using native generators:

function generate($times = 1) {
    yield 1;
    yield 2;
    yield 3;

    if ($times > 1) {
        yield from generate($times - 1);
    }
}

iterator_to_array(generate(2));

The above will return 1, 2, 3 only once.

---

The reason this happens is because every time you yield within a generator function without an explicit key, it actually yields a numerically indexed key/value pair. Every time you yield, the index gets bumped.

However, when you yield from, the inner generator yields its own indices, starting again from 0. So in my example above, it would be equivalent to this:

function generate() {
    yield 0 => 1;
    yield 1 => 2;
    yield 2 => 3;

    yield 0 => 1;
    yield 1 => 2;
    yield 2 => 3;
}

When you later convert these to an array, the later indices overwrite the earlier ones.

---

The solution is to call values() on your collection, to reset all indices:

$collection->values()->chunk(4)->toArray(); // [[1, 2, 3, 1], [2, 3]]

[driesvints]

Hey @ankr, please see the answer above.

@JosephSilber thanks!

[ankr]

@JosephSilber Thank you for the thorough explanation! It all makes sense.

[drupol]

Hi,

I made a blog post explaining this here: https://not-a-number.io/2020/lazy-collection-oddities/

I saw this thread after when I started to share the link to the post, thanks @JosephSilber for that!