Skip to content

Commit

Permalink
Programmatically scroll scroll views in the iOS 7 Simulator.
Browse files Browse the repository at this point in the history
In the iOS 7 Simulator, scroll views' pan gesture recognizers fail to cause those views
to scroll in response to UIAutomation drag gestures: a scroll view (and its pan gesture
recognizer) receives the touches involved in the gesture, but something inscrutable
goes wrong in the view's response to that gesture.

Subliminal uses a private UIAccessibility API to programmatically scroll the views
(with a higher degree of fidelity than `-setContentOffset:` would afford, e.g. the
private API notifies the delegate of appropriate scroll events as if the view
was being dragged). This method's usage need not be obfuscated because
it is compiled for the Simulator alone.
  • Loading branch information
Jeff Wear committed Oct 15, 2013
1 parent 5caeacc commit 90e6d1c
Show file tree
Hide file tree
Showing 8 changed files with 198 additions and 11 deletions.
14 changes: 9 additions & 5 deletions README.md
Expand Up @@ -189,11 +189,8 @@ Integration Tests target.

Also note that this code is conditionalized by the `INTEGRATION_TESTING` preprocessor
macro (set by `Integration Tests.xcconfig`), and so will not be built into your
main application target. Unlike many other integration testing frameworks, Subliminal
does not use private APIs, so it is safe to include calls to Subliminal APIs in
any target that links against Subliminal. However, conditionalizing calls to Subliminal
helps you keep straight exactly when you expect the tests to be run: when you
run the Integration Tests target, not every time you launch your application.
main application target. This helps you keep straight exactly when you expect the tests
to be run: when you run the Integration Tests target, not every time you launch your application.

Usage
-----
Expand Down Expand Up @@ -540,6 +537,13 @@ limitations of Apple's frameworks or bugs therein. Other issues are tracked
running iOS 5.x will fail, but dragging will succeed. Also, UIAutomation
correctly reports scroll view child elements as tappable regardless of platform.

* UIAutomation cannot drag scroll views when running in the iOS 7 Simulator.
`SLElement` implements a workaround.

> Note: The implementation of the workaround uses a private API. _However_,
poses no risk of discovery by Apple's review team (to projects linking Subliminal)
because the workaround is only compiled for the Simulator.

Contributing
------------

Expand Down
26 changes: 26 additions & 0 deletions Sources/Classes/UIAutomation/User Interface Elements/SLElement.m
Expand Up @@ -26,6 +26,7 @@
#import "SLAccessibilityPath.h"
#import "NSObject+SLVisibility.h"
#import "NSObject+SLAccessibilityDescription.h"
#import "UIScrollView+SLProgrammaticScrolling.h"


// The real value (set in `+load`) is not a compile-time constant,
Expand Down Expand Up @@ -355,6 +356,31 @@ - (void)tapAtActivationPoint {
[self waitUntilTappable:YES thenSendMessage:@"tapWithOptions({tapOffset:{x:%g, y:%g}})", activationOffset.x, activationOffset.y];
}

- (void)dragWithStartOffset:(CGPoint)startOffset endOffset:(CGPoint)endOffset {
// In the iOS 7 Simulator, scroll views' pan gesture recognizers fail to cause those views to scroll
// in response to UIAutomation drag gestures, so we must programmatically scroll those views.
// we conditionally define `kCFCoreFoundationVersionNumber_iOS_6_1` so that Subliminal
// can be continue to be built using the iOS 6.1 SDK until Travis is updated
// (https://github.com/travis-ci/travis-ci/issues/1422)
#ifndef kCFCoreFoundationVersionNumber_iOS_6_1
#define kCFCoreFoundationVersionNumber_iOS_6_1 793.00
#endif
#if TARGET_IPHONE_SIMULATOR
if (kCFCoreFoundationVersionNumber > kCFCoreFoundationVersionNumber_iOS_6_1) {
[self examineMatchingObject:^(NSObject *object) {
if ([object isKindOfClass:[UIScrollView class]]) {
[(UIScrollView *)object slScrollWithStartOffset:startOffset endOffset:endOffset];
}
}];
}
#endif

// Even if we programmatically dragged a scroll view above, we concurrently ask `UIAutomation` to drag the view,
// because scroll views(' `UIResponder` method implementations) do receive the touches involved in drag gestures
// and our programmatic scroll method does not deliver such touches.
[super dragWithStartOffset:startOffset endOffset:endOffset];
}

#pragma mark -

- (NSString *)accessibilityDescription {
Expand Down
Expand Up @@ -73,13 +73,18 @@
Informs Subliminal that this element identifies an instance of `UIScrollView`.
Developers must set this to `YES` for an element used to represent a scroll view
in tests that will be run on an iPad 5.x simulator or device, due to
[a bug in UIAutomation](-isTappable).
so that Subliminal can work around (or at the least warn of) bugs concerning
scroll views in various iOS SDK versions:
* When this is set to `YES` and tests are running on an iPad simulator or device
running iOS 5.x, Subliminal will not try to determine tappability when simulating
user interaction with that scroll view, because UIAutomation will always say
that the scroll view is not tappable.
When this is set to `YES` and tests are running on an iPad simulator or device
running iOS 5.x, Subliminal will not try to determine tappability when simulating
user interaction with that scroll view, because UIAutomation will always say
that the scroll view is not tappable.
* When this is set to `YES` and tests are running on a simulator or device (whether iPhone or iPad)
running iOS 7.x or above, Subliminal will issue a warning if it is asked to drag
the scroll view, as it will likely fail. See the documentation on `-dragWithStartOffset:endOffset:`
for more information.
Defaults to `NO`.
*/
Expand Down
Expand Up @@ -92,4 +92,24 @@ - (BOOL)isValid {
return [[[SLTerminal sharedTerminal] evalWithFormat:@"%@.isValid()", _UIARepresentation] boolValue];
}

#pragma mark -

- (void)dragWithStartOffset:(CGPoint)startOffset endOffset:(CGPoint)endOffset {
// We define `kCFCoreFoundationVersionNumber_iOS_6_1` so that Subliminal
// can be continue to be built using the iOS 6.1 SDK until Travis is updated
// (https://github.com/travis-ci/travis-ci/issues/1422)
#ifndef kCFCoreFoundationVersionNumber_iOS_6_1
#define kCFCoreFoundationVersionNumber_iOS_6_1 793.00
#endif
#if TARGET_IPHONE_SIMULATOR
if (self.isScrollView && (kCFCoreFoundationVersionNumber > kCFCoreFoundationVersionNumber_iOS_6_1)) {
NSString *warning = [NSString stringWithFormat:@"\
Dragging of %@ will most likely fail, due to a bug in iOS 7.\
See the documentation on `-dragWithStartOffset:endOffset:` for more information.", self];
[[SLLogger sharedLogger] logWarning:warning];
}
#endif
[super dragWithStartOffset:startOffset endOffset:endOffset];
}

@end
Expand Up @@ -259,6 +259,10 @@
This method uses a drag duration of 1.0 seconds (the documented default duration
for touch-and-hold gestures according to Apple's `UIAElement` class reference).
@bug On simulators running iOS 7.x, UIAutomation drag gestures do not cause
scroll views to scroll. Instances of `SLElement` are able to work around this;
instances of `SLStaticElement` are not.
@param startOffset The offset, within the element's rect, at which to begin
dragging.
@param endOffset The offset, within the element's rect, at which to end dragging.
Expand Down
@@ -0,0 +1,57 @@
//
// UIScrollView+SLProgrammaticScrolling.h
// Subliminal
//
// Created by Jeffrey Wear on 9/19/13.
// Copyright (c) 2013 Inkling. All rights reserved.
//

#import <UIKit/UIKit.h>

#if TARGET_IPHONE_SIMULATOR

/**
The methods in the `UIScrollView(SLProgrammaticScrolling)` category allow Subliminal
to programmatically scroll a `UIScrollView`. Subliminal requires this ability because,
in the iOS 7 Simulator, scroll views' pan gesture recognizers fail to respond to UIAutomation drag gestures.
NOTE: The implementation of this category uses a private API. _However_, this poses no risk
of discovery by Apple's review team (to projects linking Subliminal) because this category
is only compiled for the Simulator.
*/
@interface UIScrollView (SLProgrammaticScrolling)

/**
Scrolls the receiver by applying a relative content offset.
This method is to be used, instead of `-setContentOffset:animated:`,
because it notifies the receiver's delegate of scroll events as if a user was dragging the receiver.
Each offset specified as argument to this method describes a pair of _x_ and _y_ values,
each ranging from `0.0` to `1.0`. These values represent, respectively, relative horizontal
and vertical positions within the receiver's `-accessibilityFrame`, with `{0.0, 0.0}`
as the top left and `{1.0, 1.0}` as the bottom right.
This method will scroll with animation, but the duration of that animation
is not known or subject to Subliminal's control.
@warning This method uses an API which is only available as of iOS 7 and so must not be called
when running on iOS 6.1 or below. (It cannot be conditionally compiled for the iOS 7 SDK
because it is required by applications built using older SDKs but running on iOS 7).
@warning This method does not deliver touch events to the receiver,
thus UIAutomation's `UIAElement.dragInsideWithOptions` should be used concurrently.
@warning This method disables the receiver's `panGestureRecognizer`. (This method should be only used
in circumstances where that recognizer is non-functional anyway.)
@param startOffset The offset, within the element's accessibility frame, at which to begin dragging.
@param endOffset The offset, within the element's accessibility frame, at which to end dragging.
@exception NSInternalInconsistencyException if this method is called when running on iOS 6.1 or below.
*/
- (void)slScrollWithStartOffset:(CGPoint)startOffset endOffset:(CGPoint)endOffset;

@end

#endif
@@ -0,0 +1,63 @@
//
// UIScrollView+SLProgrammaticScrolling.m
// Subliminal
//
// Created by Jeffrey Wear on 9/19/13.
// Copyright (c) 2013 Inkling. All rights reserved.
//

#import "UIScrollView+SLProgrammaticScrolling.h"

#if TARGET_IPHONE_SIMULATOR

/*
Unlike `-[UIScrollView setContentOffset]`,
`-[UIScrollViewAccessibility(SafeCategory) accessibilityApplyScrollContent:sendScrollStatus:animated:]`
(loaded onto `UIScrollView` at runtime in iOS 7) notifies the delegate of scroll events as if the scroll view
was actually being dragged. We need to declare the method because it's a private API,
but we don't need to obscure the use of this API because this is only compiled for the Simulator.
*/
@interface UIScrollView (SLProgrammaticScrolling_Internal)

- (void)accessibilityApplyScrollContent:(CGPoint)contentOffset sendScrollStatus:(BOOL)sendStatus animated:(BOOL)animated;

@end

@implementation UIScrollView (SLProgrammaticScrolling)

- (void)slScrollWithStartOffset:(CGPoint)startOffset endOffset:(CGPoint)endOffset {
// `-[UIScrollViewAccessibility(SafeCategory) accessibilityApplyScrollContent:sendScrollStatus:animated:]`
// is only present in iOS 7
// we conditionally define `kCFCoreFoundationVersionNumber_iOS_6_1` so that Subliminal
// can be continue to be built using the iOS 6.1 SDK until Travis is updated
// (https://github.com/travis-ci/travis-ci/issues/1422)
#ifndef kCFCoreFoundationVersionNumber_iOS_6_1
#define kCFCoreFoundationVersionNumber_iOS_6_1 793.00
#endif
NSAssert(kCFCoreFoundationVersionNumber > kCFCoreFoundationVersionNumber_iOS_6_1,
@"%s is only supported on iOS 7.", __PRETTY_FUNCTION__);

CGRect accessibilityFrame = self.accessibilityFrame;
CGPoint currentContentOffset = self.contentOffset;
CGPoint newContentOffset = {
.x = currentContentOffset.x + ((startOffset.x - endOffset.x) * CGRectGetWidth(accessibilityFrame)),
.y = currentContentOffset.y + ((startOffset.y - endOffset.y) * CGRectGetHeight(accessibilityFrame))
};

// despite not ultimately causing the scroll view to scroll, the scroll view's pan gesture recognizer
// does appear to receive touches (i.e. those concurrently delivered by `UIAElement.dragInsideWithOptions`)
// and the scroll view then gets as far as calling `-[UIScrollView(UIScrollViewInternal) _scrollViewWillBeginDragging]`
// ...and then some conflict between that flow, and `-accessibilityApplyScrollContent:sendScrollStatus:animated:`
// having been called, causes an occasional crash
// so, since the gesture recognizer's not going to do the job anyway, we disable it
self.panGestureRecognizer.enabled = NO;

// I don't know what the second parameter does--either `NO` or `YES` appears to work in the Simulator
// --but it is `NO` when scrolling with VoiceOver on in an iOS device;
// we pass animated:`YES` because a user would drag with some duration.
[self accessibilityApplyScrollContent:newContentOffset sendScrollStatus:NO animated:YES];
}

@end

#endif
8 changes: 8 additions & 0 deletions Subliminal.xcodeproj/project.pbxproj
Expand Up @@ -66,6 +66,8 @@
F024BE33168BD70900708350 /* TestUtilities.m in Sources */ = {isa = PBXBuildFile; fileRef = F024BE32168BD70900708350 /* TestUtilities.m */; };
F0271AFF162E0B950098F5F2 /* SLTestController+AppHooks.h in Headers */ = {isa = PBXBuildFile; fileRef = F0271AFD162E0B950098F5F2 /* SLTestController+AppHooks.h */; settings = {ATTRIBUTES = (Public, ); }; };
F0271B00162E0B950098F5F2 /* SLTestController+AppHooks.m in Sources */ = {isa = PBXBuildFile; fileRef = F0271AFE162E0B950098F5F2 /* SLTestController+AppHooks.m */; };
F02DF30817EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.h in Headers */ = {isa = PBXBuildFile; fileRef = F02DF30617EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.h */; };
F02DF30917EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.m in Sources */ = {isa = PBXBuildFile; fileRef = F02DF30717EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.m */; };
F03C55E4170C32BE00079021 /* CoreGraphics.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = F0AC809716BB299500C5D5C0 /* CoreGraphics.framework */; };
F043469F175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.h in Headers */ = {isa = PBXBuildFile; fileRef = F043469D175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.h */; settings = {ATTRIBUTES = (Public, ); }; };
F04346A0175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.m in Sources */ = {isa = PBXBuildFile; fileRef = F043469E175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.m */; };
Expand Down Expand Up @@ -286,6 +288,8 @@
F024BE32168BD70900708350 /* TestUtilities.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = TestUtilities.m; sourceTree = "<group>"; };
F0271AFD162E0B950098F5F2 /* SLTestController+AppHooks.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "SLTestController+AppHooks.h"; sourceTree = "<group>"; };
F0271AFE162E0B950098F5F2 /* SLTestController+AppHooks.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = "SLTestController+AppHooks.m"; sourceTree = "<group>"; };
F02DF30617EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = "UIScrollView+SLProgrammaticScrolling.h"; path = "../UIAutomation/User Interface Elements/UIScrollView+SLProgrammaticScrolling.h"; sourceTree = "<group>"; };
F02DF30717EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; name = "UIScrollView+SLProgrammaticScrolling.m"; path = "../UIAutomation/User Interface Elements/UIScrollView+SLProgrammaticScrolling.m"; sourceTree = "<group>"; };
F043469D175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "NSObject+SLAccessibilityDescription.h"; sourceTree = "<group>"; };
F043469E175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = "NSObject+SLAccessibilityDescription.m"; sourceTree = "<group>"; };
F04346A5175AD10200D91F7F /* NSObject+SLVisibility.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = "NSObject+SLVisibility.h"; sourceTree = "<group>"; };
Expand Down Expand Up @@ -486,6 +490,8 @@
F04346A6175AD10200D91F7F /* NSObject+SLVisibility.m */,
F05C51E3171C8AE000A381BC /* SLMainThreadRef.h */,
F05C51E4171C8AE000A381BC /* SLMainThreadRef.m */,
F02DF30617EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.h */,
F02DF30717EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.m */,
);
path = Internal;
sourceTree = "<group>";
Expand Down Expand Up @@ -938,6 +944,7 @@
F0A04E1D1749F70F002C7520 /* SLElement.h in Headers */,
2CE9AA4C17E3A747007EF0B5 /* SLSwitch.h in Headers */,
F089F98617445D9A00DF1F25 /* SLStaticElement.h in Headers */,
F02DF30817EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.h in Headers */,
F00800CE174C1C64001927AC /* SLPopover.h in Headers */,
F043469F175ACE3A00D91F7F /* NSObject+SLAccessibilityDescription.h in Headers */,
F04346AF175AD63E00D91F7F /* SLAccessibilityPath.h in Headers */,
Expand Down Expand Up @@ -1165,6 +1172,7 @@
F0695DE8160138DF000B05D0 /* SLTest.m in Sources */,
F0695DE9160138DF000B05D0 /* SLTestController.m in Sources */,
F0271B00162E0B950098F5F2 /* SLTestController+AppHooks.m in Sources */,
F02DF30917EC064F00BE28BF /* UIScrollView+SLProgrammaticScrolling.m in Sources */,
CAC388061641CD7500F995F9 /* SLStringUtilities.m in Sources */,
CAC388401643503C00F995F9 /* NSObject+SLAccessibilityHierarchy.m in Sources */,
CA75E78516697C0000D57E92 /* SLDevice.m in Sources */,
Expand Down

0 comments on commit 90e6d1c

Please sign in to comment.