Add unlimited block unpacking

[asterite]

Fixes #8237

What's this?

This PR allows unlimited levels of block unpacking. For example, taking #8237 as a reference, this now works:

boxes = [
  { {0, 0}, {15, 10} },
  { {30, 10}, {40, 20} },
  { {9, 14}, {99, 58} },
]
boxes.each_with_index do |((x1, x2), (y1, y2)), i|
  puts "#{i}: (#{x1}, #{x2}), (#{y1}, #{y2})"
end

That's just one nested level, but it actually works with arbitrary nest levels. For example:

ary = [
  {1, {2, {3, 4}}},
]

ary.each do |(x, (y, (z, w)))|
  puts x
  puts y
  puts z
  puts w
end

Also, splats are now supported inside these unpacks!

ary = [
  [1, 2, 3, 4, 5],
]

ary.each do |(x, *y, z)|
  puts x
  puts y
  puts z
end

I believe this makes the language much more consistent with everything else.

How?

It's relevant to know how this was implemented before.

When the parser encountered something like this:

foo do |(x, y)|
end

it would actually produce an AST node that's already expanded to unpack the values.

You can see this is the case in specs, but also with this code:

macro show(node)
  {% puts node %}
end

show(foo do |(x, y)|
end)

Output:

foo do |__arg0|
  x = __arg0[0]
  y = __arg0[1]
end

That also means that up until now, a Block AST node was only able to hold a list of names (actually a list of Var nodes) as the block arguments.

Ideally, that would need to change so that each block argument can be one of two things:

1. a variable
2. an unpacked group of things, which can in turn be variables or group of things

This would require introducing a new AST node for the second point. But I believe that would be a breaking change, because AST nodes are exposed in macro code. If existing macro code relies on the fact the block arguments can only be variables, we might break things if we don't keep this guarantee.

That's why, this new implementation has the same property: the block that you get when you pass things to macros is already "expanded".

That said, the current PR improves things a bit: instead of letting the parser do the expansion, the parser will produce a Block node that has information about what to unpack. Then, right before semantic analysis, those are expanded.

Apart from the fact that user code might have relied on the fact that block arguments were only Var nodes, the existing code also relies on Block @args being an Array(Var). If we change that, we'll have to change the logic in a lot of places. That's why that didn't change: a new unpacks property was added to Block, and that is eventually put into the @args to keep backwards compatiblity.

If we eventually find that changing this representation is probably not a breaking change, we can do it. And with this PR in place it should be much simpler to do.

[HertzDevil]

The following will now break under -Dstrict_multi_assign, since the previous expansion didn't use MultiAssigns and this PR does (without the trailing *_):

{1, 2, 3}.try { |(x, y)| }

which is fine by me.

Do you have a corresponding PR for the reference manual?

[asterite]

Oh, right, the reference manual... no, I don't have a PR for that. I wonder if the manual should live in this repo too, somewhere, so that they can be kept in-sync, and we could change things in the same PR.

I won't have time to update the reference manual now, but maybe next year (I mean it)

[asterite]

Too bad this didn't get merged when I sent it. Now the merge conflict is a bit hard to fix. And this is a nice feature.

[asterite]

I'll open another PR for this.

[asterite]

Ah, it's a mess... and now with arg renamed to param. It's a lot of work 😢

If anyone wants to take a stab at this, please go ahead. I don't know why it wasn't merged back then. It's been almost a year 😢

[oprypin]

and now with arg renamed to param. It's a lot of work 😢

If anyone wants to take a stab at this,

I'm very good at merging, that part doesn't scare me, I can do it. If you merge to the commit right before the rename (git merge e05516f00~), I'll then definitely follow up and merge to latest.

[asterite]

@oprypin Thanks! I think I managed to do the merge after all. We'll see what CI says... I'll also check the final diff now.

[asterite]

I think it looks good.

This would have been nice for 1.6... 😢 I'm sure going to need this for the next Advent of Code, which I'll likely stream, trying to also use the interpreter. But I think the 1.7 release will come on January next year.

[asterite]

@straight-shoota If this turns green, is there any reason not to merge this?

[straight-shoota]

LGTM. I'm happy to merge this 👍