Merge layers taking into account z-index

[martincapello]

When a user merge layers the resulting layer should be created taking into account the z-index of the merged layers.

Originally reported here:
https://community.aseprite.org/t/merge-with-z-index/22106

[Liebranca]

The draft in 3b217b7 fixes the bug in the immediate sense, but I have some concerns about whether the way opacity is handled achieves the intended effect, so I want to take a closer look at that next. After reading app/cmd/flatten_layers.cpp I'm left thinking perhaps reusing some of that logic would be simpler in the end, but I'm not entirely sure.

[dacap]

I'm left thinking perhaps reusing some of that logic would be simpler in the end, but I'm not entirely sure

It seems to me that a "merge down" is a special case of a "flatten layers" with only two layers selected, but when more layers are selected the solution gets tricky (that's why FlattenLayers just creates a new layer to simplify the impl).

Probably we can modify FlattenLayers to improve this special case with a similar impl to "merge down" (and finally merge down could just use cmd::FlattenLayers).

[Liebranca]

I've been working on cmd::FlattenLayers in a separate branch these last few days. Because producing the final image through renderSprite fixes the issue already, I went ahead and looked at some other things around it.

My idea was addressing similar problems I noticed, that caused the result of the merge to look different to the original image in cases where both should be visually identical. Tried to fix whatever I found slightly odd while I was at it, may have gone too far in some places.

In summary:

• Changed the bool parameter for an int, so that I could pass multiple switches with one value.
• Made it so layers can be flattened "in-place", that is, overwriting the destination layer rather than spawning a new one.
• Modified the temporary image so that I could render outside the canvas. May be redundant in some cases. I went for bit of a bruteforce approach with this one, it can probably be improved.
• Changed the onExecute loop to always shrink_bounds so layer edges display properly. This one may also be redundant in some cases.
• Experimented with having z-index account for the removal of the source layers, to avoid having cels "pop" through once these layers are deleted. Doesn't work in some cases, so I have to rethink it a bit.
• Made it so opacity and blend mode are reset when flattening in-place. This is just a small detail, but it always confuses me when it isn't there.

Anyway, I have to review and clean things up, but it works. Sometime later I'd like to test if merging groups is viable with this implementation, I'm thinking it should be.

[dacap]

@Liebranca I'll take a look to your branch (I've rebased it here to main removing the merge commit). Anyway it might be better to create a PR in draft mode to review it.