fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api #277

KazuCocoa · 2020-01-26T07:21:42Z

I found the screenshot coordination should be handled as SYSTEM_VERSION_LESS_THAN(@"11.0") case, but it looked depending on the device under test environment.

I found landscape screenshot should follow SYSTEM_VERSION_LESS_THAN(@"11.0") logic on various Xcode, iOS and model versions. It meant it was difficult to say only SYSTEM_VERSION_LESS_THAN(@"11.0") or SYSTEM_VERSION_LESS_THAN(@"12.0") etc was always true in all Xcode 10, 11 environments for us (and in the future). It also could include on simulator/devices situations.

So, let me add a setting to be able to configure the coordination on the user side when they face the screenshot orientation issue.

I tested on my local.

e.g.:
On Xcode 11.3 and iPhone 11 simulator,

iOS 13 simulators returned correct screenshot coordinate without this setting
iOS 12 simulators needed this setting

mykola-mokhnach · 2020-01-29T08:43:30Z

WebDriverAgentLib/Commands/FBSessionCommands.m

@@ -43,6 +43,7 @@
 static NSString* const INCLUDE_NON_MODAL_ELEMENTS = @"includeNonModalElements";
 static NSString* const ACCEPT_ALERT_BUTTON_SELECTOR = @"acceptAlertButtonSelector";
 static NSString* const DISMISS_ALERT_BUTTON_SELECTOR = @"dismissAlertButtonSelector";
+static NSString* const SKIP_ADJUSTING_SCREENSHOT_COORDINATE = @"skipAdjustingScreenshotCoordinate";


I would say we adjust screenshot orientation rather than coordinates

mykola-mokhnach · 2020-01-29T08:44:20Z

WebDriverAgentLib/Utilities/FBConfiguration.h

+ Skip adjusting the screenshot coordinate.
+ The adjustment helps to fix the screenshot coordinate when a user change the device orientation.
+ But the logic sometimes cannot set the screenshot orientation properly. Skiping the logic can fix it.
+ Xcode versions, OS versions or device models and simulator or real device could be related.


...could influence this logic

mykola-mokhnach · 2020-01-29T14:20:32Z

WebDriverAgentLib/Commands/FBSessionCommands.m

@@ -43,6 +43,7 @@
 static NSString* const INCLUDE_NON_MODAL_ELEMENTS = @"includeNonModalElements";
 static NSString* const ACCEPT_ALERT_BUTTON_SELECTOR = @"acceptAlertButtonSelector";
 static NSString* const DISMISS_ALERT_BUTTON_SELECTOR = @"dismissAlertButtonSelector";
+static NSString* const SKIP_ADJUSTING_SCREENSHOT_ORIENTATION = @"skipAdjustingScreenshotOrientation";


how about autoAdjustScreenshotOrientation and set it to true by default?

ah. it also looks good. what about 8d033b2?

mykola-mokhnach · 2020-01-29T16:07:20Z

WebDriverAgentLib/Commands/FBSessionCommands.m

@@ -43,6 +43,7 @@
 static NSString* const INCLUDE_NON_MODAL_ELEMENTS = @"includeNonModalElements";
 static NSString* const ACCEPT_ALERT_BUTTON_SELECTOR = @"acceptAlertButtonSelector";
 static NSString* const DISMISS_ALERT_BUTTON_SELECTOR = @"dismissAlertButtonSelector";
+static NSString* const AUTO_ADJUST_SCREENSHOT_ORIENTATION = @"autoAdjustScreenshotOrientation";


I think we could make it more flexible for clients by explicitly providing the appropriate orientation value, like UIImageOrientationLeft or UIImageOrientationUp and leaving it to auto by default

mykola-mokhnach · 2020-01-30T06:25:25Z

WebDriverAgentLib/Commands/FBSessionCommands.m

@@ -328,6 +329,12 @@ + (NSArray *)routes
    [FBConfiguration setDismissAlertButtonSelector:(NSString *)[settings objectForKey:DISMISS_ALERT_BUTTON_SELECTOR]];
  }

+#if !TARGET_OS_TV


mykola-mokhnach · 2020-01-30T06:25:52Z

WebDriverAgentLib/Utilities/FBConfiguration.h

+ But the logic sometimes cannot set the screenshot orientation properly. Skiping the logic can fix it.
+ Xcode versions, OS versions or device models and simulator or real device could influence this logic.
+
+ @param orientation Set to NO in order to skip adjusting screenshot coordinate. Defaults to YES.


Please update the docstring

mykola-mokhnach · 2020-01-30T06:27:13Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+  // Only UIInterfaceOrientationUnknown is over iOS 8. Others are over iOS 2.
+  // https://developer.apple.com/documentation/uikit/uiinterfaceorientation/uiinterfaceorientationunknown
+  if (orientation == nil) {
+    FBScreenshotOrientation = UIInterfaceOrientationUnknown;


this could be merged with else block

WebDriverAgentLib/Utilities/FBConfiguration.m

mykola-mokhnach · 2020-01-30T06:45:14Z

WebDriverAgentLib/Utilities/FBImageUtils.m

-  } else if (orientation == UIInterfaceOrientationPortraitUpsideDown) {
-    imageOrientation = UIImageOrientationDown;
+  // Apply auto rotation by default for iOS >= 11.0. In iOS < 11.0 screenshots are already adjusted properly.
+  if (FBConfiguration.screenshotOrientation == UIInterfaceOrientationUnknown && SYSTEM_VERSION_GREATER_THAN_OR_EQUAL_TO(@"11.0")) {


I'd rather change this condition to always apply the current approach if the screenshotOrientation is set to auto. Otherwise forcefully apply the method set by the client without any additional checks

…string for users

mykola-mokhnach · 2020-01-30T16:52:11Z

WebDriverAgentLib/Utilities/FBConfiguration.h

+/**
+@return The orientation as String for human read
+*/
+ (NSString *)screenshotOrientationForUser;


humanReadableScreenshotOrientation

mykola-mokhnach · 2020-01-30T16:52:52Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+  } else if (FBScreenshotOrientation == UIInterfaceOrientationLandscapeLeft) {
+     return @"LandscapeLeft";
+  }
+  return @"Auto";


lets make them starting with lowercase, like the other existing setting names

mykola-mokhnach · 2020-01-30T16:53:14Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+
+ (NSString *)screenshotOrientationForUser
+{
+  if (FBScreenshotOrientation == UIInterfaceOrientationPortrait) {


You could use switch

mykola-mokhnach · 2020-01-30T16:55:05Z

WebDriverAgentLib/Utilities/FBImageUtils.m

-  } else if (orientation == UIInterfaceOrientationPortraitUpsideDown) {
-    imageOrientation = UIImageOrientationDown;
+  if (FBConfiguration.screenshotOrientation == UIInterfaceOrientationUnknown) {
+    if (orientation == UIInterfaceOrientationLandscapeRight) {


if (SYSTEM_VERSION_LESS_THAN(@"11.0")) { is missing here. The previous implementation should be kept unchanged

mykola-mokhnach · 2020-01-30T16:56:08Z

WebDriverAgentLib/Utilities/FBImageUtils.m

+      imageOrientation = UIImageOrientationDown;
+    } else if (FBConfiguration.screenshotOrientation == UIInterfaceOrientationLandscapeRight) {
+      imageOrientation = UIImageOrientationLeft;
+    }else if (FBConfiguration.screenshotOrientation == UIInterfaceOrientationLandscapeLeft) {


missing space

mykola-mokhnach · 2020-01-30T16:56:52Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+    FBScreenshotOrientation = UIInterfaceOrientationPortraitUpsideDown;
+  } else if ([orientation.lowercaseString isEqualToString:@"landscaperight"]) {
+    FBScreenshotOrientation = UIInterfaceOrientationLandscapeRight;
+  }else if ([orientation.lowercaseString isEqualToString:@"landscapeleft"]) {


missing space

mykola-mokhnach · 2020-01-30T16:57:15Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+  }else if ([orientation.lowercaseString isEqualToString:@"landscapeleft"]) {
+    FBScreenshotOrientation = UIInterfaceOrientationLandscapeLeft;
+  } else {
+    FBScreenshotOrientation = UIInterfaceOrientationUnknown;


I'd rather throw an error if an unknown setting value is provided

mykola-mokhnach · 2020-01-31T10:52:22Z

WebDriverAgentLib/Utilities/FBConfiguration.m

-     return @"landscapeRight";
-  } else if (FBScreenshotOrientation == UIInterfaceOrientationLandscapeLeft) {
-     return @"landscapeLeft";
+  switch (FBScreenshotOrientation) {


mykola-mokhnach · 2020-01-31T10:53:18Z

WebDriverAgentLib/Commands/FBSessionCommands.m

+    NSError *error;
+    if (![FBConfiguration setScreenshotOrientation:(NSString *)[settings objectForKey:SCREENSHOT_ORIENTATION]
+                                             error:&error]) {
+      return FBResponseWithStatus([FBCommandStatus invalidArgumentErrorWithMessage:error.description traceback:nil]);


mykola-mokhnach · 2020-01-31T10:53:50Z

WebDriverAgentLib/Utilities/FBConfiguration.h

+
+ @param orientation Set the orientation to adjust the screenshot.
+ Case insensitive "portrait", "portraitUpsideDown", "landscapeRight" and "landscapeLeft"  are available
+ to force the coodinate adjust. Other wards are handled as "auto", which handles


mykola-mokhnach · 2020-01-31T10:55:25Z

WebDriverAgentLib/Utilities/FBConfiguration.m

+    FBScreenshotOrientation = UIInterfaceOrientationUnknown;
+  } else {
+    return [[FBErrorBuilder.builder withDescriptionFormat:
+             @"No available orientation strategies. Available strategies are 'auto', 'portrate', " \


I would rather say The orientation value '%@' is not known. Only the following orientation values are supported: %@

fix: make adjusting screenshot coordinate configurable

2ad9289

KazuCocoa requested review from mykola-mokhnach and imurchie January 29, 2020 00:08

mykola-mokhnach reviewed Jan 29, 2020

View reviewed changes

KazuCocoa added 2 commits January 29, 2020 19:15

Rename

8e1d1a0

fix compile error

84bf3ff

mykola-mokhnach reviewed Jan 29, 2020

View reviewed changes

rename, change the default value

8d033b2

mykola-mokhnach reviewed Jan 29, 2020

View reviewed changes

KazuCocoa added 2 commits January 30, 2020 14:00

set an arbitrary orientation

0a3c626

Use UIInterfaceOrientation

e77a936

mykola-mokhnach reviewed Jan 30, 2020

View reviewed changes

WebDriverAgentLib/Utilities/FBConfiguration.m Show resolved Hide resolved

mykola-mokhnach reviewed Jan 30, 2020

View reviewed changes

tweak docstring, add screenshotOrientationForUser to return readable …

b78445a

…string for users

mykola-mokhnach reviewed Jan 30, 2020

View reviewed changes

KazuCocoa added 2 commits January 31, 2020 10:50

Add error response

9440004

use switch for UIInterfaceOrientation

963b3ab

mykola-mokhnach reviewed Jan 31, 2020

View reviewed changes

mykola-mokhnach approved these changes Jan 31, 2020

View reviewed changes

Tweak messages

7edcaaa

KazuCocoa merged commit c529906 into appium:master Jan 31, 2020

KazuCocoa deleted the km/add-skiping-screenshot-coordinate branch January 31, 2020 17:10

KazuCocoa changed the title ~~fix: make adjusting screenshot coordinate configurable~~ fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api Feb 1, 2020

This was referenced Feb 1, 2020

docs: add intentOptions for espresso and screenshotOrientation for xcuitest appium/appium#13898

Merged

Bump up xcuitest patch version appium/appium#13915

Closed

KazuCocoa mentioned this pull request Mar 3, 2020

changelog: add 1.17.0 CHANGELOG appium/appium#14015

Merged

renovate bot mentioned this pull request Jun 6, 2021

Update dependency appium to v1.22.0 tadashi0713/webdriverio-appium-on-docker-android#15

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api #277

fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api #277

KazuCocoa commented Jan 26, 2020 •

edited

Loading

mykola-mokhnach Jan 29, 2020

mykola-mokhnach Jan 29, 2020

mykola-mokhnach Jan 29, 2020

KazuCocoa Jan 29, 2020

mykola-mokhnach Jan 29, 2020

KazuCocoa Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 30, 2020

mykola-mokhnach Jan 31, 2020

mykola-mokhnach Jan 31, 2020

mykola-mokhnach Jan 31, 2020

mykola-mokhnach Jan 31, 2020 •

edited

Loading

fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api #277

fix: make adjusting screenshot coordinate configurable - screenshotOrientation settings api #277

Conversation

KazuCocoa commented Jan 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mykola-mokhnach Jan 31, 2020 • edited Loading

Choose a reason for hiding this comment

KazuCocoa commented Jan 26, 2020 •

edited

Loading

mykola-mokhnach Jan 31, 2020 •

edited

Loading