[iOS 17] Software keyboard overlaps position: fixed; text fields near…

… the bottom of the viewport https://bugs.webkit.org/show_bug.cgi?id=258828 rdar://109127515 Reviewed by Aditya Keerthi. On iOS 17, in the case where out-of-process keyboard is enabled and the software keyboard is shown, we end up getting two sets of `"KeyboardWillShow"` -> `"KeyboardDidShow"` notifications when the keyboard animates up, after reloading input views. When the first set of notifications is dispatched underneath the call to `-reloadInputViews`, the keyboard hasn't yet become full height, and so our attempts to zoom to reveal the focused element using the current input view bounds will fail. I filed <rdar://111704216> for UIKit to investigate restoring pre-iOS-17 behavior, with respect to dispatching these intermediate keyboard notifications; in the meantime, this patch works around this behavior change by deferring the call to `-_zoomToRevealFocusedElement` until the first `"KeyboardWillShow"` notification arrives *after* `-reloadInputViews` has already been invoked, in the case where OOP keyboard is enabled and the keyboard is full height. We limit this to the case of OOP keyboard because when that feature is disabled, we receive a "KeyboardWillShow" notification underneath the call to `-reloadInputViews` that already contains the final keyboard height; furthermore, we limit this to the case where the keyboard is full height, because when OOP keyboard is enabled and the keyboard is minimized (e.g. the hardware keyboard is attached), we only get a single set of keyboard appearance notifications. * LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input-expected.txt: Added. * LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input.html: Added. Add a new layout test to exercise keyboard scrolling, in the case where: 1. We're showing the software keyboard. 2. The focused element that would be overlapped by the keyboard is in a fixed position container. 3. The page itself is scrollable. ...and verify that we successfully scroll such that the bottom of the caret rect is above the top of the keyboard (input view bounds). * LayoutTests/resources/ui-helper.js: (window.UIHelper.getUICaretViewRectInGlobalCoordinates.return.new.Promise.): (window.UIHelper.getUICaretViewRectInGlobalCoordinates.return.new.Promise): (window.UIHelper.getUICaretViewRectInGlobalCoordinates): Add support for a new `UIHelper` method to grab the selection caret rect in web view coordinates. The "global" here is consistent terminology used elsewhere in `UIScriptController`. * Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm: (-[WKContentView _elementDidFocus:userIsInteracting:blurPreviousNode:activityStateChanges:userObject:]): See comments above for more details. * Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl: * Tools/TestRunnerShared/UIScriptContext/UIScriptController.h: (WTR::UIScriptController::selectionCaretViewRectInGlobalCoordinates const): * Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h: * Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm: (WTR::UIScriptControllerIOS::selectionCaretViewRect const): (WTR::UIScriptControllerIOS::selectionCaretViewRectInGlobalCoordinates const): Canonical link: https://commits.webkit.org/265741@main
WebKit · Jul 4, 2023 · 6669d12 · 6669d12
1 parent a4a461f
commit 6669d12
Show file tree

Hide file tree

Showing 8 changed files with 149 additions and 9 deletions.
diff --git a/LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input-expected.txt b/LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input-expected.txt
@@ -0,0 +1,10 @@
+
+This test verifies that we properly reveal focused elements in fixed position containers which would otherwise be obscured by the software keyboard. To reproduce, tap on the input field below. The software keyboard should not obscure the selection caret.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+PASS inputViewBounds.top is >= caretRect.top + caretRect.height
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input.html b/LayoutTests/fast/forms/ios/scroll-to-reveal-fixed-input.html
@@ -0,0 +1,80 @@
+<!DOCTYPE html> <!-- webkit-test-runner [ useFlexibleViewport=true ] -->
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<head>
+<script src="../../../resources/js-test.js"></script>
+<script src="../../../resources/ui-helper.js"></script>
+<style>
+    html, body {
+        width: 100%;
+        height: 100%;
+        color: lightgray;
+        font-size: 20px;
+    }
+
+    body {
+        margin: 0;
+        text-align: center;
+        font-family: system-ui;
+        background-color: #222;
+        overflow: scroll;
+    }
+
+    .tall {
+        width: 100%;
+        height: 200vh;
+    }
+
+    .container {
+        position: fixed;
+        width: 100%;
+        height: 100%;
+    }
+
+    input {
+        border: none;
+        border-radius: 4px;
+        outline: none;
+        font-size: 20px;
+        background-color: lightgray;
+        position: absolute;
+        bottom: 0;
+        left: 0;
+        width: 100%;
+        padding: 8px;
+    }
+</style>
+<script>
+jsTestIsAsync = true;
+
+addEventListener("load", async () => {
+    description(`This test verifies that we properly reveal focused elements in fixed position
+        containers which would otherwise be obscured by the software keyboard. To reproduce, tap on
+        the input field below. The software keyboard should not obscure the selection caret.`);
+
+    await UIHelper.setHardwareKeyboardAttached(false);
+
+    const textField = document.querySelector("input");
+    await UIHelper.activateElementAndWaitForInputSession(textField);
+
+    caretRect = await UIHelper.getUICaretViewRectInGlobalCoordinates();
+    inputViewBounds = await UIHelper.inputViewBounds();
+
+    shouldBeGreaterThanOrEqual("inputViewBounds.top", "caretRect.top + caretRect.height");
+
+    textField.blur();
+    await UIHelper.waitForKeyboardToHide();
+    finishJSTest();
+});
+</script>
+</head>
+<body>
+<div class="container">
+    <input autofocus placeholder="Tap here"></input>
+</div>
+<div class="tall"></div>
+<div id="description"></div>
+<div id="console"></div>
+</body>
+
+</html>
diff --git a/LayoutTests/resources/ui-helper.js b/LayoutTests/resources/ui-helper.js
@@ -896,6 +896,22 @@ window.UIHelper = class UIHelper {
         });
     }
 
+    static getUICaretViewRectInGlobalCoordinates()
+    {
+        if (!this.isWebKit2() || !this.isIOSFamily())
+            return Promise.resolve();
+
+        return new Promise(resolve => {
+            testRunner.runUIScript(`(function() {
+                uiController.doAfterNextStablePresentationUpdate(function() {
+                    uiController.uiScriptComplete(JSON.stringify(uiController.selectionCaretViewRectInGlobalCoordinates));
+                });
+            })()`, jsonString => {
+                resolve(JSON.parse(jsonString));
+            });
+        });
+    }
+
     static getUISelectionViewRects()
     {
         if (!this.isWebKit2() || !this.isIOSFamily())

diff --git a/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm b/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
@@ -7245,11 +7245,26 @@ - (void)_elementDidFocus:(const WebKit::FocusedElementInformation&)information u
     // selection rect before we can zoom and reveal the selection. Non-selectable elements (e.g. <select>) can be zoomed
     // immediately because they have no selection to reveal.
     if (requiresKeyboard) {
+        bool ignorePreviousKeyboardWillShowNotification = [] {
+            // In the case where out-of-process keyboard is enabled and the software keyboard is shown,
+            // we end up getting two sets of "KeyboardWillShow" -> "KeyboardDidShow" notifications when
+            // the keyboard animates up, after reloading input views. When the first set of notifications
+            // is dispatched underneath the call to -reloadInputViews above, the keyboard hasn't yet
+            // become full height, so attempts to reveal the focused element using the current height will
+            // fail. The second set of keyboard notifications contains the final keyboard height, and is the
+            // one we should use for revealing the focused element.
+            // See also: <rdar://111704216>.
+            if (!UIKeyboard.usesInputSystemUI)
+                return false;
+
+            auto keyboard = UIKeyboard.activeKeyboard;
+            return keyboard && !keyboard.isMinimized;
+        }();
         _revealFocusedElementDeferrer = WebKit::RevealFocusedElementDeferrer::create(self, [&] {
             OptionSet reasons { WebKit::RevealFocusedElementDeferralReason::EditorState };
             if (!self._scroller.firstResponderKeyboardAvoidanceEnabled)
                 reasons.add(WebKit::RevealFocusedElementDeferralReason::KeyboardDidShow);
-            else if (_waitingForKeyboardAppearanceAnimationToStart)
+            else if (_waitingForKeyboardAppearanceAnimationToStart || ignorePreviousKeyboardWillShowNotification)
                 reasons.add(WebKit::RevealFocusedElementDeferralReason::KeyboardWillShow);
             return reasons;
         }());

diff --git a/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl b/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
@@ -320,12 +320,14 @@ interface UIScriptController {
     // When false, content rect updates to the web process have inStableState=false, as if a scroll or zoom were in progress.
     attribute boolean? stableStateOverride;
 
-    readonly attribute object contentVisibleRect; // Returned object has 'left', 'top', 'width', 'height' properties.
-
+    // The attributes below return an array of objects with 'left', 'top', 'width', and 'height' properties.
+    readonly attribute object contentVisibleRect;
     readonly attribute object selectionStartGrabberViewRect;
     readonly attribute object selectionEndGrabberViewRect;
-    readonly attribute object selectionCaretViewRect; // An array of objects with 'left', 'top', 'width', and 'height' properties.
-    readonly attribute object selectionRangeViewRects; // An object with 'left', 'top', 'width', 'height' properties.
+    readonly attribute object selectionCaretViewRect;
+    readonly attribute object selectionCaretViewRectInGlobalCoordinates;
+    readonly attribute object selectionRangeViewRects;
+
     readonly attribute object calendarType;
     undefined setDefaultCalendarType(DOMString calendarIdentifier, DOMString localeIdentifier);
     readonly attribute object inputViewBounds;

diff --git a/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h b/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
@@ -309,6 +309,7 @@ class UIScriptController : public JSWrappable {
     virtual JSObjectRef selectionStartGrabberViewRect() const { notImplemented(); return nullptr; }
     virtual JSObjectRef selectionEndGrabberViewRect() const { notImplemented(); return nullptr; }
     virtual JSObjectRef selectionCaretViewRect() const { notImplemented(); return nullptr; }
+    virtual JSObjectRef selectionCaretViewRectInGlobalCoordinates() const { notImplemented(); return nullptr; }
     virtual JSObjectRef selectionRangeViewRects() const { notImplemented(); return nullptr; }
 
     // Rotation

diff --git a/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h b/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h
@@ -33,6 +33,8 @@
 typedef struct CGRect CGRect;
 OBJC_CLASS UITextSelectionDisplayInteraction;
 
+@protocol UICoordinateSpace;
+
 namespace WebCore {
 class FloatPoint;
 class FloatRect;
@@ -115,6 +117,8 @@ class UIScriptControllerIOS final : public UIScriptControllerCocoa {
     JSObjectRef selectionStartGrabberViewRect() const override;
     JSObjectRef selectionEndGrabberViewRect() const override;
     JSObjectRef selectionCaretViewRect() const override;
+    JSObjectRef selectionCaretViewRectInGlobalCoordinates() const override;
+    JSObjectRef selectionCaretViewRect(id<UICoordinateSpace>) const;
     JSObjectRef selectionRangeViewRects() const override;
     JSObjectRef inputViewBounds() const override;
     JSRetainPtr<JSStringRef> scrollingTreeAsText() const override;

diff --git a/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm b/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm
@@ -882,7 +882,7 @@ static void clipSelectionViewRectToContentView(CGRect& rect, UIView *contentView
     return JSValueToObject(jsContext, [JSValue valueWithObject:toNSDictionary(frameInContentViewCoordinates) inContext:[JSContext contextWithJSGlobalContextRef:jsContext]].JSValueRef, nullptr);
 }
 
-JSObjectRef UIScriptControllerIOS::selectionCaretViewRect() const
+JSObjectRef UIScriptControllerIOS::selectionCaretViewRect(id<UICoordinateSpace> coordinateSpace) const
 {
     UIView *contentView = platformContentView();
     UIView *caretView = nil;
@@ -894,9 +894,21 @@ static void clipSelectionViewRectToContentView(CGRect& rect, UIView *contentView
     caretView = [contentView valueForKeyPath:@"interactionAssistant.selectionView.caretView"];
 #endif
 
-    auto rectInContentViewCoordinates = CGRectIntersection([caretView convertRect:caretView.bounds toView:contentView], contentView.bounds);
-    clipSelectionViewRectToContentView(rectInContentViewCoordinates, contentView);
-    return JSValueToObject(m_context->jsContext(), [JSValue valueWithObject:toNSDictionary(rectInContentViewCoordinates) inContext:[JSContext contextWithJSGlobalContextRef:m_context->jsContext()]].JSValueRef, nullptr);
+    auto contentRect = CGRectIntersection([caretView convertRect:caretView.bounds toView:contentView], contentView.bounds);
+    clipSelectionViewRectToContentView(contentRect, contentView);
+    if (coordinateSpace != contentView)
+        contentRect = [coordinateSpace convertRect:contentRect fromCoordinateSpace:contentView];
+    return JSValueToObject(m_context->jsContext(), [JSValue valueWithObject:toNSDictionary(contentRect) inContext:[JSContext contextWithJSGlobalContextRef:m_context->jsContext()]].JSValueRef, nullptr);
+}
+
+JSObjectRef UIScriptControllerIOS::selectionCaretViewRect() const
+{
+    return selectionCaretViewRect(platformContentView());
+}
+
+JSObjectRef UIScriptControllerIOS::selectionCaretViewRectInGlobalCoordinates() const
+{
+    return selectionCaretViewRect(webView());
 }
 
 JSObjectRef UIScriptControllerIOS::selectionRangeViewRects() const