Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(): selection logic to support nested multiselection #8665

Merged
merged 59 commits into from
Feb 18, 2023
Merged

Conversation

ShaMan123
Copy link
Contributor

@ShaMan123 ShaMan123 commented Feb 5, 2023

Motivation

  • fix selection logic to support nested multiselection + object stacking control on ActiveSelection
  • make selection logic more scoped and orderly (I have future thoughts making it an interaction brush so scoping logic is great), it can be extracted once this PR is merged easily to a standalone manager
  • make canvas scoped so in the future stuff can be extracted to standalone modules and it can be reduced in size and complexity

ports and closes #7878
ports logic from #7882 not including rendering logic that will be done after this PR

closes #6840
closes #4529

Description

Selection

This PR refactors what was known as canvas grouping logic (formerly the canvas grouping mixin).
We are talking about a bunch of methods that were in charge of selecting objects in response to user events.
Selection is distinct into 2.
What is known as multiselection (shift+click) and what is known as selection/group selection (drag) defined as _groupSelector (the blue rect that selects objects)
Logic was scattered and unscoped (methods shared logic, e.g #8665 (comment)) for some reason, what made this effort difficult, confusing and time consuming. This is why I squashed them as follows:

I updated the logic to act with the constant active selection ref (see below) - #8665 (comment).
So basically the result of this PR is 2 methods that are solely in charge of selection logic.
I decided to rename the main methods because:

  • naming was unclear (I still think it is, suggestions are welcome)
  • since I squashed the condition to run the logic into the main method, whoever is using it directly (poor unfortunate fellow) might experience a break of the method being skipped because of the condition and not understanding why. However these are private methods so I regret to tell this dev, tough luck, or actually good fortune, since now I think it is much readable and therefore reliable.
  • renaming make a breaking change so less breaking. It breaks the app but shows up everywhere and then the dev must read migration docs and they see what happened and how to move forward as opposed of having their app continue working but then bugs appear and edge cases etc.

Active Selection Ref

Made canvas hold a constant ref to active selection. This is done to allow devs to listen to events and handle it as they wish, what was impossible until now (you could subscribe after an active selection was created). It also allows to subclass active selection and make canvas use it.
#8665 (comment)

  • replaced most of isActiveSelection calls with a simple obj === canvas._activeSelection

MultiSelection Stacking Order

  • exposed multiSelectStacking: 'canvas-stacking' | 'selection-order' that controls how multiselected objects are added to the active selection. Logic is handled by ActiveSelection#multiSelectAdd
  • fixed the bug from _updateActiveSelection
  • refactored findTarget(e, skipGroup) => FabricObject => findTarget(e) => FabricObject: skipGroup is not needed anymore, it was buggy so now calling searchPossibleTarget directly and scoped internal logic

30ee763

Other Changes

  • fixed ancestry methods to comply with active selection in the tree
  • simplified _chooseObjectsToRender
  • renamed _groupSelector keys for clarity 58c050d
  • added a minimal rendering change to Group in order to test the PR
  • removed dead prop _previousPointer
  • disabled node assertion config fd145fe, it was passing false positives

Gist

TO DO

In Action

@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2023

Build Stats

file / KB (diff) bundled minified
fabric 896.716 (+4.352) 304.667 (+0.554)

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2023

Coverage after merging v6-selection into master will be

83.08%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
fabric.ts100%100%100%100%
index.node.ts7.69%100%0%14.29%18, 21, 24, 30, 33, 36
src
   Collection.ts96.28%96.43%90%98.04%206–207, 232–233
   CommonMethods.ts100%100%100%100%
   Intersection.ts100%100%100%100%
   Observable.ts87.23%85.29%84.62%89.36%144–145, 170–171, 39–40, 48, 57, 91, 99
   Pattern.ts95.77%94.29%100%96.43%124, 135, 144
   Point.ts100%100%100%100%
   Shadow.ts98.48%96%100%100%156
   cache.ts97.06%90%100%100%56
   config.ts75%64.29%66.67%82.76%138, 138, 138, 138, 138–140, 151–153
   constants.ts100%100%100%100%
   typedefs.ts100%100%100%100%
src/brushes
   BaseBrush.ts100%100%100%100%
   CircleBrush.ts0%0%0%0%100, 102–104, 113, 113, 113, 115, 117, 119–121, 123–126, 133–134, 141, 143, 28–29, 37–41, 45–49, 56–59, 67–71, 73, 81, 81, 81, 81, 81–82, 84, 84, 84–87, 89, 97–98
   PatternBrush.ts97.06%87.50%100%100%21
   PencilBrush.ts91.01%82.35%100%93.75%122–123, 152, 152–154, 176, 176, 276, 280, 285–286, 68–69, 84–85
   SprayBrush.ts0%0%0%0%100–101, 103–104, 112, 112, 112, 112, 112–113, 115–116, 123–124, 126, 128–132, 141, 145–146, 146, 154, 154, 154–157, 159–162, 166–167, 169, 171–174, 177, 184–185, 187, 189–190, 192, 199–200, 202–203, 206, 206, 213, 213, 217, 22–23, 25–27, 27, 27–29, 33, 42, 49, 56, 63, 70, 89–91, 99
src/canvas
   Canvas.ts78.93%77.41%81.67%79.60%1000, 1000, 1000–1002, 1004–1005, 1005, 1005, 1007, 1015, 1015, 1015–1017, 1017, 1017, 1023–1024, 1032–1033, 1033, 1033–1034, 1039, 1041, 1072–1074, 1077–1078, 1082–1083, 1197–1199, 1202–1203, 1276, 1396, 1519, 1589, 161, 186, 295–296, 299–303, 308, 331–332, 337–342, 36, 362, 362, 362–363, 363, 363–364, 372, 377–378, 378, 378–379, 381, 390, 396–397, 397, 397, 40, 440, 448, 452, 452, 452–453, 455, 537–538, 538, 538–539, 545, 545, 545–547, 567, 569, 569, 569–570, 570, 570, 573, 573, 573–574, 577, 586–587, 589–590, 592, 592–593, 595–596, 608–609, 609, 609–610, 612–616, 622, 629, 669, 669, 669, 671, 673–677, 683, 689, 689, 689–690, 692, 695, 700, 713, 740, 740, 798–799, 799, 799–800, 802, 805–806, 806, 806–807, 809–810, 813, 813–815, 818–819, 889, 901, 908, 929, 961, 982–983, 999
   SelectableCanvas.ts94.15%91.16%94.34%96.26%1102, 1102–1103, 1106, 1126, 1126, 1185, 1235–1236, 1257, 1265, 1385–1386, 1388–1389, 1410–1411, 681–682, 684–685, 685, 685, 734–735, 796–797, 850–852, 884, 889–890, 917–918
   StaticCanvas.ts95.96%91.29%100%97.92%1083–1084, 1084, 1084–1085, 1205, 1215, 1269–1270, 1273, 1308–1309, 1386, 1395, 1395, 1399, 1399, 1446–1447, 1661–1662, 282–283, 300, 380–381, 383–384, 745, 757–758, 843
   TextEditingManager.ts100%100%100%100%
src/color
   Color.ts92.16%86.49%100%94.29%330–331, 335–336, 339–340, 58, 88–89, 89, 91, 91, 91–92, 94–95
   color_map.ts100%100%100%100%
   constants.ts100%100%100%100%
   util.ts100%100%100%100%
src/controls
   Control.ts93.90%88.89%90.91%97.73%235, 319, 319, 354
   changeWidth.ts100%100%100%100%
   controls.render.ts81.63%78%100%84.78%106, 111, 121, 121, 45, 50, 61, 61, 65–72, 81–82
   default_controls.ts86.67%66.67%100%100%122, 129
   drag.ts100%100%100%100%
   polyControl.ts6.15%0%0%11.11%100, 105, 119, 119, 119, 121–124, 124, 127, 134, 17, 25–28, 30, 30, 30, 30, 30, 30, 30, 30, 50–56, 56, 56, 56, 56, 58, 63–64, 66, 76, 82–83, 83, 83–84, 88–90, 90, 90, 90, 90, 92
   rotate.ts19.57%12.50%50%21.43%41, 45, 51, 51, 51–52, 55–57, 59, 59, 59, 59, 59–61, 61, 61–63, 65, 65, 65–67, 67, 67–68, 73, 73, 73–74, 76, 78, 80–81
   scale.ts93.49%92.77%100%93.67%129–130, 132–134, 148–149, 181–183, 42
   scaleSkew.ts78.79%64.29%100%85.71%27, 29, 29, 29, 31, 33, 35
   skew.ts91.03%79.31%100%97.67%130–131, 162–163, 170, 176, 178
   util.ts100%100%100%100%
   wrapWithFireEvent.ts100%100%100%100%
   wrapWithFixedAnchor.ts100%100%100%100%
src/env
   browser.ts84.21%77.78%50%100%14, 17
   index.ts93.75%50%100%100%20
   node.ts76.92%50%66.67%88.24%27, 31–32, 32, 32
src/filters
   BaseFilter.ts19.77%20%30%17.65%100–101, 101, 101–102, 109–112, 112, 112–113, 119, 119, 119–122, 140, 170–175, 179–180, 180, 180–183, 183, 183, 183,

@ShaMan123 ShaMan123 changed the title fix(): selection changes to support nested multiselection fix(): selection logic to support nested multiselection Feb 6, 2023
Copy link
Contributor Author

@ShaMan123 ShaMan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good

const shouldRender = this._shouldRender(target),
shouldGroup = this._shouldGroup(e, target);
const shouldRender = this._shouldRender(target);
let shouldGroup = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do we know that this was always false in all possible combinations?
If this is a micro optimization ( don't run should group if i should clear selection ) i would not do it.
or i would carefully check the commit history before changing it.

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this very closely.
Let me explain.

shouldGroup = this._shouldGroup(e, target);
if (this._shouldClearSelection(e, target)) {
  this.discardActiveObject(e);
} else if (shouldGroup) {
  // in order for shouldGroup to be true, target needs to be true
  this._handleGrouping(e, target!);
  target = this._activeObject;
}

_shouldClearSelection and _shouldGroup are opposites of each other


!!target &&

The whole point of this change is to squash them together. The logic is too complex and scattered around everywhere

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so when you agree I will squash them here

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/canvas/Canvas.ts Outdated Show resolved Hide resolved
Copy link
Member

@asturur asturur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert the shouldGroup boolean detection order if it there isn't a specific case that is fixing or needs it.

Comment on lines -1935 to -1959
QUnit.test('active group objects reordering', function(assert) {
var rect1 = new fabric.Rect({ width: 30, height: 30, left: 130, top: 130 });
var rect2 = new fabric.Rect({ width: 50, height: 50, left: 100, top: 100 });
var circle1 = new fabric.Circle({ radius: 10, left: 60, top: 60 });
var circle2 = new fabric.Circle({ radius: 50, left: 50, top: 50 });
canvas.add(rect1, rect2, circle1, circle2);
assert.equal(canvas._objects[0], rect1);
assert.equal(canvas._objects[1], rect2);
assert.equal(canvas._objects[2], circle1);
assert.equal(canvas._objects[3], circle2);
var aGroup = new fabric.ActiveSelection([rect2, circle2, rect1, circle1], { canvas: canvas });
// before rendering objects are ordered in insert order
assert.equal(aGroup._objects[0], rect2);
assert.equal(aGroup._objects[1], circle2);
assert.equal(aGroup._objects[2], rect1);
assert.equal(aGroup._objects[3], circle1);
canvas.setActiveObject(aGroup)
canvas.renderAll();
// after rendering objects are ordered in canvas stack order
assert.equal(aGroup._objects[0], rect1);
assert.equal(aGroup._objects[1], rect2);
assert.equal(aGroup._objects[2], circle1);
assert.equal(aGroup._objects[3], circle2);
});

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asturur please explain this to me as part of the discussion of click order vs. stack order
I believe this is what convinced me it was a bug, that rendering changed the order of the objects in active selection to the canvas stacking order.
Now with this PR active selection will respect both click order and stack order

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say before it didn't respect either

@ShaMan123
Copy link
Contributor Author

ShaMan123 commented Feb 8, 2023

No it didn't, it captured them in click order. Now it seems that it capture in stack order. But why one is wrong and one is good?

Originally posted by @asturur in #8665 (comment)

@asturur you were wrong, it never was click order. It was canvas stacking order always. Regardless I have promoted the fix to support both and then I added tests and saw this.

QUnit.test('create active selection selected in different order, but same result', function(assert) {
var isFired = false;
var rect1 = new fabric.Rect();
var rect2 = new fabric.Rect();
canvas.add(rect1, rect2);
rect2.on('selected', function( ) { isFired = true; });
canvas.setActiveObject(rect2);
canvas._createActiveSelection({}, rect1);
const activeSelection = canvas.getActiveObjects();
assert.equal(activeSelection[0], rect1, 'first rec1');
assert.equal(activeSelection[1], rect2, 'then rect2');
canvas.clear();
});

fix selecting active selection as part of a `n` multi selection click
add `multiSelectionStacking`, `multiSelectAdd`
Copy link
Contributor Author

@ShaMan123 ShaMan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright
Now it is done
You can resolve any conversation you like. I used it to navigate the code

Updated description. I had a fatal error, mixed up the names of the new methods and add more about the stack ordering solution
I also added tests that should cover all cases of handleMultiSelect

Comment on lines +1492 to +1496
// find target from active objects
target = this.searchPossibleTargets(
prevActiveObjects,
this.getPointer(e, true)
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of this.findTarget(e, true)

Comment on lines 941 to 942
aObjects.length > 1 &&
isActiveSelection(activeObject) &&
!skipGroup &&
activeObject === this._activeSelection &&
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this condition is a bit silly because if aObjects > 1 we are dealing with the active selection but I think you put that for TS

@ShaMan123
Copy link
Contributor Author

and of course something is wrong

Copy link
Contributor Author

@ShaMan123 ShaMan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now with findTarget fixed and tested this is done
Was a hard one

@asturur
Copy link
Member

asturur commented Feb 11, 2023

No it didn't, it captured them in click order. Now it seems that it capture in stack order. But why one is wrong and one is good?

Originally posted by @asturur in #8665 (comment)

@asturur you were wrong, it never was click order. It was canvas stacking order always. Regardless I have promoted the fix to support both and then I added tests and saw this.

QUnit.test('create active selection selected in different order, but same result', function(assert) {
var isFired = false;
var rect1 = new fabric.Rect();
var rect2 = new fabric.Rect();
canvas.add(rect1, rect2);
rect2.on('selected', function( ) { isFired = true; });
canvas.setActiveObject(rect2);
canvas._createActiveSelection({}, rect1);
const activeSelection = canvas.getActiveObjects();
assert.equal(activeSelection[0], rect1, 'first rec1');
assert.equal(activeSelection[1], rect2, 'then rect2');
canvas.clear();
});

The multiple selection is in click order.
After you select the first 2 obects, all the others were added in the active selectin inside the array as last one, and that is visible in the bug you linked.

On top of that i can't see anymore the activeSelection.toGroup method. what happened to it?

@asturur
Copy link
Member

asturur commented Feb 11, 2023

ok toGroup and toActiveSelection have been killed in the group transition for some reason, i will add them back in a separate pr using whatever code we have now to do a similar thing

@asturur
Copy link
Member

asturur commented Feb 11, 2023

@ShaMan123 cloned this branch locally, i will be offline 2 days reviewing the new selection process.

@ShaMan123
Copy link
Contributor Author

ShaMan123 commented Feb 11, 2023

have been killed in the group transition for some reason, i will add them back in a separate pr using whatever code we have now to do a similar thing

I documented this I think.
There is no reason for this any longer because there is no overhead logic performed.

new Group(canvas.getActiveObjects())

Why would we expose a method for this?

@ShaMan123
Copy link
Contributor Author

ShaMan123 commented Feb 11, 2023

And to add to active selection

(canvas.getActiveObject() || canvas.getActiveSelection()).add(...group.objects)

That is the whole point of a strict object tree

@ShaMan123
Copy link
Contributor Author

ShaMan123 commented Feb 11, 2023

The multiple selection is in click order.
After you select the first 2 obects, all the others were added in the active selectin inside the array as last one, and that is visible in the bug you linked.

For me this is inconsistent behavior. First it does stack order and then click order. That is why I considered it a bug when I saw it.

Another thing I would like to understand from you is why does the multiselection logic runs on mouse down and not up

Comment on lines +44 to +64
/**
* Adds objects with respect to {@link multiSelectionStacking}
* @param targets object to add to selection
*/
multiSelectAdd(...targets: FabricObject[]) {
if (this.multiSelectionStacking === 'selection-order') {
this.add(...targets);
} else {
// respect object stacking as it is on canvas
// perf enhancement for large ActiveSelection: consider a binary search of `isInFrontOf`
targets.forEach((target) => {
const index = this._objects.findIndex((obj) => obj.isInFrontOf(target));
const insertAt =
index === -1
? // `target` is in front of all other objects
this.size()
: index;
this.insertAt(insertAt, target);
});
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why calling this add and not using the fact that this is a subclass of group, exactly to handle multi selections, and so its own add method is either super.add or this.insertAt?
I would gladly rename this in add. Maybe i ll merge this as it is, and i ll just open a PR with the things i wouldn't want to do

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought of that as well.
wasn't sure what I/you think about that.
The reason I am not for is in case the dev wants to change the stack order, using insertAt is valuable so if we change it it can be annoying.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but anyone can do this._objects.insertAt so I am ok with it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do it

Copy link
Contributor Author

@ShaMan123 ShaMan123 Feb 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this will be add and insertAt will be left as is?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was just saying to rename multiSelectAdd in add.
Because this class has to handle only multi selection logic and nothing else. so seems legit that its own add is special.

Are you thinking that people may want to call add of an active selection directly to do things in the active selection outside the canvas flow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes
Perhaps
What do you think?

Comment on lines +487 to +494
if (
this.canvas?.preserveObjectStacking &&
this._objects[i].group !== this
) {
ctx.save();
ctx.transform(...invertTransform(this.calcTransformMatrix()));
this._objects[i].render(ctx);
ctx.restore();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the edge case handling, this is exactly how you do it.
The only improvement would be that instead of going back with an invert and a calc transform, you go straight to viewport transform using ctx.setTransform(this.canvas.viewportTransform).
I will add it in a follow up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this logic should be handled by active selection.
We should discuss this, there are a lot of implications.

@asturur
Copy link
Member

asturur commented Feb 18, 2023

@ShaMan123 as is i m merging this because there are many improvements here compared to the things i want to remove.
But i will need to come back with a PR ( like right away ) to remove/imporve the things i think aren't related to fixing the selection and are also new complexities i don't want to deal with.

  • activeSelection.toGroup and group.toActiveSelection need to come back
  • _activeSelection and getActiveSelection are a new behaviour i don't think is necessary and so i don't want to introduce
    all the rest seems good to me

@asturur asturur merged commit 224f407 into master Feb 18, 2023
@ShaMan123 ShaMan123 deleted the v6-selection branch February 19, 2023 05:16
@ShaMan123
Copy link
Contributor Author

activeSelection.toGroup and group.toActiveSelection need to come back

why?

new Group(canvas.getActiveObjects()) // toGroup 
(canvas.getActiveObject() || canvas.getActiveSelection()).add(...group.objects) // toActiveSelection 

#8665 (comment)
#8665 (comment)

@ShaMan123 ShaMan123 linked an issue Feb 19, 2023 that may be closed by this pull request
@asturur
Copy link
Member

asturur commented Feb 20, 2023

They were simply part of the api, not deprecated and useful, and have been removed in a larger PR without noticing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants