Using Detox with custom accessibilityElements

[TheLartians]

Hey, first of all thanks for this great tool!

I am currently trying to use Detox to replace Xcode's UI tests, however having trouble working with custom views defining their own accessibilityElements. In my case I'm using an GLKView view that overrides the accessibilityElements method for obtaining displayed inner elements. For example, I'm rendering a full Keypad in a single GLKView and returning an accessibility element for every visible key.

This works fine for automated UITests using XCTest and also all elements are discoverable by id in Detox using element(by.id(id)). However, interacting with elements, e.g. calling element(by.id(id)).tap() fails in Detox due to not passing the visibility threshold.

Test Failed: View “<RNTMaphiView: 0x7fa6bcee28c0>” is not visible: view does not pass visibility threshold (0% visible of 75% required)

Using the accessibility inspector and Xcode seems to confirm that the element is correctly placed and at the top of the view hierarchy and not blocked by other elements.

I've also ran the app using -detoxDebugVisibility YES , which does not seem to show any components blocking the view, though it also doesn't show the expected frame of the button (in this case f(x) in the lower left).

The accessible elements are created in Objective C using

UIAccessibilityElement *uielement = [[UIAccessibilityElement alloc] initWithAccessibilityContainer:self];
uielement.accessibilityIdentifier = element.id;
uielement.isAccessibilityElement = true;
uielement.accessibilityFrameInContainerSpace = CGRect{
  {element.frame.xmin(), element.frame.ymin()},
  {element.frame.width(), element.frame.height()}
};
uielement.accessibilityFrameInContainerSpace = frame;
uielement.accessibilityTraits |= UIAccessibilityTraitButton;

Not really sure what the problem is. Is detox compatible with custom accessibility elements? Any other approaches I can test?

[support]

We use the issue tracker exclusively for bug reports and feature requests. This issue appears to be a general usage or support question. Instead, please ask a question on Stack Overflow with the detox tag.

Feel free to post your Stack Overflow question here for more visibility. We'll take a look at it.

For issues with Expo apps, it is most likely not an issue with Detox itself, but with the Expo runtime or with incorrect Detox setup. For support on how to use Detox with Expo, you should contact the Expo team or the Expo community.

For more information on bots in this repository, read this discussion.

[TheLartians]

It turns out that Detox sees the elements, but misses the coordinates by a factor of two. I've solved the issue for Detox by overloading accessibilityElementAtIndex in my Views subclass:

- (id) accessibilityElementAtIndex:(NSInteger)index {
  if (detox) {
    UIAccessibilityElement * element = [super accessibilityElementAtIndex:index];
    if (element) {
      UIAccessibilityElement * copy = [[UIAccessibilityElement alloc] initWithAccessibilityContainer:self];
      copy.isAccessibilityElement = element.isAccessibilityElement;
      copy.accessibilityTraits = element.accessibilityTraits;
      copy.accessibilityIdentifier = element.accessibilityIdentifier;
      auto frame = element.accessibilityFrameInContainerSpace;
      frame.origin.x /= 2;
      frame.origin.y /= 2;
      frame.size.width /= 2;
      frame.size.height /= 2;
      copy.accessibilityFrameInContainerSpace = frame;
      return copy;
    }
    return nil;
  } else {
    return [super accessibilityElementAtIndex:index];
  }
}

By providing a launch argument for running in Detox, my project works both for standard iOS accessibility and Detox use.