Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transforms break UI hit testing #18408

Closed
wmleler opened this issue Jun 12, 2018 · 10 comments
Closed

Transforms break UI hit testing #18408

wmleler opened this issue Jun 12, 2018 · 10 comments
Assignees
Milestone

Comments

@wmleler
Copy link

@wmleler wmleler commented Jun 12, 2018

If you use a Transform widget to perform a 3D transformation, it breaks the UI (you cannot click on a button anymore). Scaling and rotateZ seem to be fine, but rotateX or rotateY seem to break things if you rotate more than a few degrees.

To reproduce, run the file https://gist.github.com/wmleler/cfc18137ce7db77d7e599e179dee4619 (change its name to main.dart, of course)

To rotate things, just touch the screen and pan around, like this (the little moving circle is the position of the user's finger):

screen2

As long as the screen is not rotated very much, you can still tap the FAB and increment the counter. Rotate a little more and the tap target is slightly away from the FAB itself and you can still find it by tapping around. But rotate more than that, and you can't even find the tap target.

@wmleler

This comment has been minimized.

Copy link
Author

@wmleler wmleler commented Jun 13, 2018

Here's a question on StackOverflow with the same problem -- https://stackoverflow.com/questions/50541250/transform-gesture-detector-in-flutterwith-stack

@wmleler

This comment has been minimized.

Copy link
Author

@wmleler wmleler commented Jun 13, 2018

Does "transformHitTests" property actually do anything? It seems to make no difference in my app.

@dnfield

This comment has been minimized.

Copy link
Member

@dnfield dnfield commented Jun 15, 2018

I was suspicious that this could be a Path.getBounds issue, but that doesn't seem to be the culprit here. It looks like the hit testing transformation just isn't quite working right.

By default, transformHitTests is set to true. If you set it to false, you'll be able to tap your button where it originally was living (e.g. if no transform had been applied). When it's set to true, the hit position is getting translated, but it's not quite getting translated correctly - and it doesn't look like the hittable area is actually getting translated either.

I'm still working through a couple things - looks like there was some attempt to fix this in #16558, but looks like it still isn't quite right.

@wmleler

This comment has been minimized.

Copy link
Author

@wmleler wmleler commented Jun 15, 2018

@dnfield

This comment has been minimized.

Copy link
Member

@dnfield dnfield commented Jun 15, 2018

With it set to false, I'm able to make the button go by tapping where it would be if no transforms were applied... i.e. in the bottom right corner of the screen (where the red dot is):

image

I've added some code to paint a red dot where the transform is making the hit test go to. The best I can make of it is that the logic (which looks like it should make sense) is actually mapping the physical device tap to the perspective transformed image correctly, but the hit testing logic still thinks the button is where it originally lived. It seems like this could be fixed by either transforming the point or transforming the hit test area. It's not very obvious to me as to how to do either though.

@wmleler

This comment has been minimized.

Copy link
Author

@wmleler wmleler commented Jun 15, 2018

You're right. I was fooled because it turns out that when I change transformHitTests to false, it is not good enough to do either a hot reload or a full reload. I have to stop the app from running and restart it from scratch in order to get that change to take effect. Once I did that, setting transformHitTests to false does make the hit target stay where it was before the transform took effect.

If transformHitTests is true (or not specified of course) then as you rotate the scaffold slightly so that the bottom is further away then the top, the hit target does move, but it moves downward. So it is not at the same place as the FAB. If you rotate further, then the hit target is no longer on the screen.

@chanyap92

This comment has been minimized.

Copy link

@chanyap92 chanyap92 commented Jan 28, 2019

I have 2 Buttons inside a Stack, but after i do the transform animation the Green button is unable to trigger onPressed event. I try so many approach but no luck, if I replace the Stack with Column then the problem gone, but I must use stack for this UI

untitled

import 'dart:math';

import 'package:flutter/material.dart';

class TestPage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        floatingActionButtonLocation: FloatingActionButtonLocation.centerDocked,
        floatingActionButton: FancyFab(),
        bottomNavigationBar: BottomAppBar(
          shape: CircularNotchedRectangle(),
          notchMargin: 4.0,
          child: Row(),
          elevation: 7,
        ));
  }
}

class FancyFab extends StatefulWidget {
  @override
  _FancyFabState createState() => _FancyFabState();
}

class _FancyFabState extends State<FancyFab>
    with SingleTickerProviderStateMixin {
  bool isOpened = false;
  AnimationController _animationController;
  Animation<Color> _buttonColor;
  Animation<double> _animateIcon;
  Animation<double> _translateButton;
  Curve _curve = Curves.easeInOut;
  double actionRadius = 80;

  @override
  initState() {
    _animationController =
        AnimationController(vsync: this, duration: Duration(milliseconds: 500))
          ..addListener(() {
            setState(() {});
          });
    _animateIcon =
        Tween<double>(begin: 0.0, end: 1.0).animate(_animationController);
    _buttonColor = ColorTween(
      begin: Colors.blue,
      end: Colors.red,
    ).animate(CurvedAnimation(
      parent: _animationController,
      curve: Interval(
        0.00,
        1.00,
        curve: Curves.linear,
      ),
    ));
    _translateButton = Tween<double>(
      begin: 0,
      end: 1,
    ).animate(CurvedAnimation(
      parent: _animationController,
      curve: Interval(
        0.0,
        0.5,
        curve: Curves.linear,
      ),
    ));
    super.initState();
  }

  @override
  dispose() {
    _animationController.dispose();
    super.dispose();
  }

  animate() {
    if (!isOpened) {
      _animationController.forward();
    } else {
      _animationController.reverse();
    }
    isOpened = !isOpened;
  }

  @override
  Widget build(BuildContext context) {
    return Stack(
      children: <Widget>[
        Transform(
            transform: Matrix4.translationValues(
              _translateButton.value * actionRadius * cos(4.5 * pi / 4),
              _translateButton.value * actionRadius * sin(4.5 * pi / 4),
              0.0,
            ),
            child: FloatingActionButton(
                backgroundColor: Colors.green,
                onPressed: () {
                  //This code never fire
                  print('test print');
                },
                child: Icon(Icons.cached))),
        FloatingActionButton(
          backgroundColor: _buttonColor.value,
          onPressed: animate,
          child: AnimatedIcon(
            icon: AnimatedIcons.menu_close,
            progress: _animateIcon,
          ),
        ),
      ],
    );
  }
}
@HemilKumbhani

This comment has been minimized.

Copy link

@HemilKumbhani HemilKumbhani commented Apr 10, 2019

Use container above Stack and give it height and width
Here the issue is after the transformation the Button is moved while the stack is still at the same place so the device is not able to detect the touch event.
Now your code will be as below.

  Widget build(BuildContext context) {
    return Container(
      width: 300,
      height: 300,
      child: Stack(
        children: <Widget>[
          Transform(
              transform: Matrix4.translationValues(
                _translateButton.value * actionRadius * cos(4.5 * pi / 4),
                _translateButton.value * actionRadius * sin(4.5 * pi / 4),
                0.0,
              ),
              child: FloatingActionButton(
                  backgroundColor: Colors.green,
                  onPressed: () {
                    //This code never fire
                    print('test print');
                  },
                  child: Icon(Icons.cached))),
          FloatingActionButton(
            backgroundColor: _buttonColor.value,
            onPressed: animate,
            child: AnimatedIcon(
              icon: AnimatedIcons.menu_close,
              progress: _animateIcon,
            ),
          ),
        ],
      ),
    );
  }`
@spkersten

This comment has been minimized.

Copy link
Contributor

@spkersten spkersten commented May 18, 2019

I found the problem when the transformation contains out of plane rotations (i.e. around the x or y axis). The problem is that RenderTransform.hitTestChildren uses MatrixUtils.transformPoint(inverse, position) when hit at position, where inverse is the inverse of the transformation matrix.

MatrixUtils.transformPoint gets a 2D-point and assumes the z coordinate is zero:

static Offset transformPoint(Matrix4 transform, Offset point) {
    final Vector3 position3 = Vector3(point.dx, point.dy, 0);
    final Vector3 transformed3 = transform.perspectiveTransform(position3);
    print("z_transformed = ${transformed3.z}");
    return Offset(transformed3.x, transformed3.y);
  }

That is wrong in this case. The original child of Transformed is flat and has z = 0. After a transformation that contains out of plane rotations, z_transformed != 0. That z is ignored, as it is irrelevant for painting. However, when transforming back, as is the cases for hit testing, it should be taking into account. After the inverse transformation, z = 0 should hold again, since we should have transformed the point back onto the z = 0 plane. That is not the case, as will be evident by the print statement I added above.

The correct inverse transformation has to recover the "lost" transformed z before applying the matrix. That z is easily found by using the fact that the inverse transformed z must be zero, that is, by solving 0 = A_2 * x + A_6 * y + A_10 * Z + A_14. The proper inverse transform of a 2D point becomes:

static Offset inverseTransformPoint(Matrix4 inverseTransform, Offset point) {
    final A = inverseTransform.storage;
    final Z = -(A[2]*point.dx + A[6]*point.dy + A[14])/A[10];
    final Vector3 position3 = Vector3(point.dx, point.dy, Z);
    final Vector3 transformed3 = inverseTransform.perspectiveTransform(position3);
    print("z = ${transformed3.z} (should be zero)");
    return Offset(transformed3.x, transformed3.y);
  }

I ignored the cases where A[10] == 0 (a π/2 rotation around x or y axis) but that should be handled separately.

(There is also an inverseTransformRect that, from code inspection, appears to have the same bug.)

@Hixie

This comment has been minimized.

Copy link
Contributor

@Hixie Hixie commented May 18, 2019

@spkersten You may be interested in reviewing PR #32192. Your feedback there would be most welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.