Transforms break UI hit testing

[wmleler]

If you use a Transform widget to perform a 3D transformation, it breaks the UI (you cannot click on a button anymore). Scaling and rotateZ seem to be fine, but rotateX or rotateY seem to break things if you rotate more than a few degrees.

To reproduce, run the file https://gist.github.com/wmleler/cfc18137ce7db77d7e599e179dee4619 (change its name to main.dart, of course)

To rotate things, just touch the screen and pan around, like this (the little moving circle is the position of the user's finger):

As long as the screen is not rotated very much, you can still tap the FAB and increment the counter. Rotate a little more and the tap target is slightly away from the FAB itself and you can still find it by tapping around. But rotate more than that, and you can't even find the tap target.

[wmleler]

Here's a question on StackOverflow with the same problem -- https://stackoverflow.com/questions/50541250/transform-gesture-detector-in-flutterwith-stack

[wmleler]

Does "transformHitTests" property actually do anything? It seems to make no difference in my app.

[dnfield]

I was suspicious that this could be a Path.getBounds issue, but that doesn't seem to be the culprit here. It looks like the hit testing transformation just isn't quite working right.

By default, transformHitTests is set to true. If you set it to false, you'll be able to tap your button where it originally was living (e.g. if no transform had been applied). When it's set to true, the hit position is getting translated, but it's not quite getting translated correctly - and it doesn't look like the hittable area is actually getting translated either.

I'm still working through a couple things - looks like there was some attempt to fix this in #16558, but looks like it still isn't quite right.

[wmleler]

I did set transformHitTests to false, and it didn't seem to make any difference at all. I could find the hit position in the same place it was when transformHitTests was true and not where the FAB was before being transformed. [this is not true... see below]

…

On Thu, Jun 14, 2018 at 5:17 PM, Dan Field ***@***.***> wrote: I was suspicious that this could be a Path.getBounds issue, but that doesn't seem to be the culprit here. It looks like the hit testing transformation just isn't quite working right. By default, transformHitTests is set to true. If you set it to false, you'll be able to tap your button where it originally was living (e.g. if no transform had been applied). When it's set to true, the hit position is getting translated, but it's not quite getting translated correctly - and it doesn't look like the hittable area is actually getting translated either. I'm still working through a couple things - looks like there was some attempt to fix this in #16558 <#16558>, but looks like it still isn't quite right. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#18408 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADEtSQjytKf2uHWCHcCXN7A8U-XxNeCks5t8v0ogaJpZM4Uk1kd> .

[dnfield]

With it set to false, I'm able to make the button go by tapping where it would be if no transforms were applied... i.e. in the bottom right corner of the screen (where the red dot is):

I've added some code to paint a red dot where the transform is making the hit test go to. The best I can make of it is that the logic (which looks like it should make sense) is actually mapping the physical device tap to the perspective transformed image correctly, but the hit testing logic still thinks the button is where it originally lived. It seems like this could be fixed by either transforming the point or transforming the hit test area. It's not very obvious to me as to how to do either though.

[wmleler]

You're right. I was fooled because it turns out that when I change transformHitTests to false, it is not good enough to do either a hot reload or a full reload. I have to stop the app from running and restart it from scratch in order to get that change to take effect. Once I did that, setting transformHitTests to false does make the hit target stay where it was before the transform took effect.

If transformHitTests is true (or not specified of course) then as you rotate the scaffold slightly so that the bottom is further away then the top, the hit target does move, but it moves downward. So it is not at the same place as the FAB. If you rotate further, then the hit target is no longer on the screen.

[chanyap92]

I have 2 Buttons inside a Stack, but after i do the transform animation the Green button is unable to trigger onPressed event. I try so many approach but no luck, if I replace the Stack with Column then the problem gone, but I must use stack for this UI

import 'dart:math';

import 'package:flutter/material.dart';

class TestPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        floatingActionButtonLocation: FloatingActionButtonLocation.centerDocked,
        floatingActionButton: FancyFab(),
        bottomNavigationBar: BottomAppBar(
          shape: CircularNotchedRectangle(),
          notchMargin: 4.0,
          child: Row(),
          elevation: 7,
        ));
  }
}

class FancyFab extends StatefulWidget {
  @override
  _FancyFabState createState() => _FancyFabState();
}

class _FancyFabState extends State<FancyFab>
    with SingleTickerProviderStateMixin {
  bool isOpened = false;
  AnimationController _animationController;
  Animation<Color> _buttonColor;
  Animation<double> _animateIcon;
  Animation<double> _translateButton;
  Curve _curve = Curves.easeInOut;
  double actionRadius = 80;

  @override
  initState() {
    _animationController =
        AnimationController(vsync: this, duration: Duration(milliseconds: 500))
          ..addListener(() {
            setState(() {});
          });
    _animateIcon =
        Tween<double>(begin: 0.0, end: 1.0).animate(_animationController);
    _buttonColor = ColorTween(
      begin: Colors.blue,
      end: Colors.red,
    ).animate(CurvedAnimation(
      parent: _animationController,
      curve: Interval(
        0.00,
        1.00,
        curve: Curves.linear,
      ),
    ));
    _translateButton = Tween<double>(
      begin: 0,
      end: 1,
    ).animate(CurvedAnimation(
      parent: _animationController,
      curve: Interval(
        0.0,
        0.5,
        curve: Curves.linear,
      ),
    ));
    super.initState();
  }

  @override
  dispose() {
    _animationController.dispose();
    super.dispose();
  }

  animate() {
    if (!isOpened) {
      _animationController.forward();
    } else {
      _animationController.reverse();
    }
    isOpened = !isOpened;
  }

  @override
  Widget build(BuildContext context) {
    return Stack(
      children: <Widget>[
        Transform(
            transform: Matrix4.translationValues(
              _translateButton.value * actionRadius * cos(4.5 * pi / 4),
              _translateButton.value * actionRadius * sin(4.5 * pi / 4),
              0.0,
            ),
            child: FloatingActionButton(
                backgroundColor: Colors.green,
                onPressed: () {
                  //This code never fire
                  print('test print');
                },
                child: Icon(Icons.cached))),
        FloatingActionButton(
          backgroundColor: _buttonColor.value,
          onPressed: animate,
          child: AnimatedIcon(
            icon: AnimatedIcons.menu_close,
            progress: _animateIcon,
          ),
        ),
      ],
    );
  }
}

[HemilKumbhani]

Use container above Stack and give it height and width
Here the issue is after the transformation the Button is moved while the stack is still at the same place so the device is not able to detect the touch event.
Now your code will be as below.

  Widget build(BuildContext context) {
    return Container(
      width: 300,
      height: 300,
      child: Stack(
        children: <Widget>[
          Transform(
              transform: Matrix4.translationValues(
                _translateButton.value * actionRadius * cos(4.5 * pi / 4),
                _translateButton.value * actionRadius * sin(4.5 * pi / 4),
                0.0,
              ),
              child: FloatingActionButton(
                  backgroundColor: Colors.green,
                  onPressed: () {
                    //This code never fire
                    print('test print');
                  },
                  child: Icon(Icons.cached))),
          FloatingActionButton(
            backgroundColor: _buttonColor.value,
            onPressed: animate,
            child: AnimatedIcon(
              icon: AnimatedIcons.menu_close,
              progress: _animateIcon,
            ),
          ),
        ],
      ),
    );
  }`

[spkersten]

I found the problem when the transformation contains out of plane rotations (i.e. around the x or y axis). The problem is that RenderTransform.hitTestChildren uses MatrixUtils.transformPoint(inverse, position) when hit at position, where inverse is the inverse of the transformation matrix.

MatrixUtils.transformPoint gets a 2D-point and assumes the z coordinate is zero:

static Offset transformPoint(Matrix4 transform, Offset point) {
    final Vector3 position3 = Vector3(point.dx, point.dy, 0);
    final Vector3 transformed3 = transform.perspectiveTransform(position3);
    print("z_transformed = ${transformed3.z}");
    return Offset(transformed3.x, transformed3.y);
  }

That is wrong in this case. The original child of Transformed is flat and has z = 0. After a transformation that contains out of plane rotations, z_transformed != 0. That z is ignored, as it is irrelevant for painting. However, when transforming back, as is the cases for hit testing, it should be taking into account. After the inverse transformation, z = 0 should hold again, since we should have transformed the point back onto the z = 0 plane. That is not the case, as will be evident by the print statement I added above.

The correct inverse transformation has to recover the "lost" transformed z before applying the matrix. That z is easily found by using the fact that the inverse transformed z must be zero, that is, by solving 0 = A_2 * x + A_6 * y + A_10 * Z + A_14. The proper inverse transform of a 2D point becomes:

static Offset inverseTransformPoint(Matrix4 inverseTransform, Offset point) {
    final A = inverseTransform.storage;
    final Z = -(A[2]*point.dx + A[6]*point.dy + A[14])/A[10];
    final Vector3 position3 = Vector3(point.dx, point.dy, Z);
    final Vector3 transformed3 = inverseTransform.perspectiveTransform(position3);
    print("z = ${transformed3.z} (should be zero)");
    return Offset(transformed3.x, transformed3.y);
  }

I ignored the cases where A[10] == 0 (a π/2 rotation around x or y axis) but that should be handled separately.

(There is also an inverseTransformRect that, from code inspection, appears to have the same bug.)

[Hixie]

@spkersten You may be interested in reviewing PR #32192. Your feedback there would be most welcome.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.