Sub descend and method Any::descend return a recursively flat Seq

[gfldex]

of anything that implements deepmap.

[zoffixznet]

What's the usecase for this method/sub? I rather not add more core methods and subs that save only half a dozen of keystrokes, unless they are very commonly used. Especially since we already have flat that has the same result, at least for Seqs

[gfldex]

see discussion https://irclog.perlgeek.de/perl6/2016-07-28#i_12929268

flat simply does not work for most use cases. It doesn't work for arrays nor does it recurse array or not. We have deepmap and map to recurse or not but there is no recursive iterator. descend fills that gap.

[zoffixznet]

As I've mentioned on IRC, I feel like maybe .flat could use an improvement rather than us adding a whole new routine for this functionality. Is there, perhaps, an argument we could add instead? :deeply?

Maybe flat should function like the proposed descend instead of the current behavior and we'd want to implement that in 6.d instead? Personally, I expected say flat [1,[2,3,[4,5,6]]] to give a single flat list, and IIRC I even opened a ticket for that, thinking flat was buggy when it didn't do that.

Lastly, if we do go with adding a new routine, I think the name is not ideal. Currently it implies going into the guts of the thing and doing something while in there, whereas it just flattens the thing (which is what drove my feeling that we should be improving flat instead).

My choice would be to make flat behave like proposed descend.

[gfldex]

flat makes a distinction between Array and List/Seq.

my @a = flat <a b> Z, ( (1, 2) Z, (3, 4) ); dd @a;
OUTPUT«Array @a = ["a", 1, 3, "b", 2, 4]␤»
my @a = flat <a b> Z, [ (1, 2) Z, (3, 4) ]; dd @a;
OUTPUT«Array @a = ["a", (1, 3), "b", (2, 4)]␤»

Just changing the behaviour of flat will have massive fallout in the ecosystem. Adding a named to flat should not.

[zoffixznet]

Just changing the behaviour of flat will have massive fallout in the ecosystem

How did you arrive at that estimate?

Looking at May 3rd ecosystem snapshot, the about 380 instances of flat's usage are heavily dominated by flattening ranges, results of xx operators in lists, and superstitious flattening. None of that will be affected by the change, and it will be a 6.d change too.

I'm hardpressed to find code that relies on flat not flattenning arrays or to think of a usecase where that is desired. The behaviour would still be attainable with a few extra keystrokes—say, map *.Slip,—and flat would do what its name says it'll do: make stuff flat.

[ab5tract]

I'm curious what TimToady has to say, in that this is not the first time it has come up. Thus, there must have been a reason provided for the current behavior. I'd like to interrogate that reasoning while discussing this change.

[smls]

FYI, I've written up a summary of the current options for "deep" (Scalar-descending) flattening in this Stack Overflow answer.

TL;DR:

• gather @a.deepmap: *.take is the simplest existing solution.
• Once .[**] is implemented, it should be able to do the same.
  ^{(Except it would returning a List rather than Seq, but if you're working with arrays you're probably doing imperative programming anyway, rather than the kind of functional programming that benefits the most from memory-efficient lazy infinite sequences.)}.
• In the other answer/comments on that page, people seem to like the name deepflat for this.

---

I'm hardpressed to find code that relies on flat not flattenning arrays or to think of a usecase where that is desired.

I've used it in code golf, but probably not in "real" code. (I could be wrong, of course - it might be insightful to change the behavior in a Rakudo branch and see what breaks.)

---

Thus, there must have been a reason provided for the current behavior.

Before the GLR, lots of things (e.g. map, array variables, etc.) flattened incoming lists automatically.

Therefore, people needed a way to prevent sub-lists from being flattened in case they wanted to work with nested data-structures. Scalar-ization was used for this because it fits the idea of "$=singular, @=plural", and because it's similar to how things work in Perl 5 (with the analogy "container=reference").

The GLR changed a lot, and now built-ins prefer to keep nested lists as they are, and instead let you manually flatten them when when you do want it - either from the outside (flat), or from the inside (slip).

However, the release of 6.c happened shortly after the GLR was finished in order to make the Christmas deadline, and I think not all parts of the language were thoroughly reviewed to re-examine them in light of the GLR.
^{(Note that when Larry announced/predicted the 6.c Christmas release in January 2015, the GLR was expected to be done in spring, but it ended up being done much later in the year, and the release date was kept anyway for publicity reasons I guess.)}

If there had been more time and willingness to reexamined/change things after the GLR, my personal guess is that the whole concept of Scalar preventing its value from being treated as an Iterable might have been abolished. (Maybe it would have even been renamed Scalar-->Variable, accordingly).

It's not just flat where this behavior of Scalars gets in the way more than it helps now. For example, initializing objects from Hashes ^{(which one might want do for many reasons, e.g. if one gets the data from a JSON parser that outputs hashes)} is kind of a hazard now:

class Person { has $.name; has @.hobbies; }
my %a = name => "Ben", hobbies => <reading skating>;
say Person.new(|%a);
# The @.hobbies attribute now doesn't hold the two elements "reading"
# and "skating" as the user probably expected, but rather the single
# element $("reading", "skating")

In practice, scalarization is still sometimes used to force an array to be treated as a single item, but I can't think of any case where it is necessary, because there are alternative ways to achieve the same thing. For example:

say $@array X 1..10;    # (@array, 1), (@array, 2), ... (@array, 10)
say (@array,) X 1..10;  # Scalarization-free alternative

So... that's where things are at.

Whether it is possible to change any of this (either specifically for flat, or more generally), I don't know. Maybe as an opt-in change when you write use 6.d;?
That's up to TimToady and Zoffix & co. to decide / figure out... :)

[zoffixznet]

gather @a.deepmap: *.take is the simplest existing solution.

Note that it ruins Maps:

<Zoffix> m: dd gather [%(a => 42, b => 72), 99].deepmap: *.take
<camelia> rakudo-moar 7f245f: OUTPUT«(42, 72, 99).Seq␤»

[zoffixznet]

Thank you for your effort, however I'm going to reject this. It adds another method to practically all Perl 6 classes that saves only three words of typing and doesn't work with Hashes and Maps in a desirable way.