[iOS] Adjust some viewport behaviors when multitasking mode is enabled

https://bugs.webkit.org/show_bug.cgi?id=240151
rdar://87157773

Reviewed by Tim Horton.

Add a new ViewportConfiguration flag to prefer horizontal scrolling over shrinking to fit when the view layout
size falls under the current "default desktop webpage" of 980pt. See WebKit changes for more details.

* page/ViewportConfiguration.cpp:
(WebCore::ViewportConfiguration::initialScaleFromSize const):
(WebCore::ViewportConfiguration::webpageParameters):
(WebCore::ViewportConfiguration::imageDocumentParameters):

Pull the magic value representing the "assumed width of most desktop webpages" (980) out into a named constant,
`defaultDesktopViewportWidth`, so that we can consult it when computing the initial scale.

(WebCore::ViewportConfiguration::description const):
* page/ViewportConfiguration.h:
(WebCore::ViewportConfiguration::setPrefersHorizontalScrollingBelowDesktopViewportWidths):
[iOS] Adjust some viewport behaviors when multitasking mode is enabled
https://bugs.webkit.org/show_bug.cgi?id=240151
rdar://87157773

Reviewed by Tim Horton.

Make some adjustments to viewport behaviors when multitasking mode is enabled. See the comments below for more
details. There are no changes in behavior when multitasking mode is disabled; tests for the new behaviors in
multitasking mode will be added in a subsequent patch.

* Shared/WebPageCreationParameters.cpp:
(WebKit::WebPageCreationParameters::encode const):
(WebKit::WebPageCreationParameters::decode):
* Shared/WebPageCreationParameters.h:

Add plumbing to inform the web process when "multitasking mode" state changes; we use this bit in WebPage to
determine whether or not we should use multitasking mode viewport behaviors (see below).

* UIProcess/API/ios/WKWebViewIOS.mm:
(-[WKWebView _registerForNotifications]):
(-[WKWebView didMoveToWindow]):
(-[WKWebView _multitaskingModeDidChange:]):

Send IPC to WebPage in these two places, to keep "multitasking mode" state in sync with the native view.

(-[WKWebView _beginAnimatedResizeWithUpdates:]):

Make a minor adjustment here to ignore `oldWebViewWidthInContentViewCoordinates` when computing a target scale
to zoom to when performing animated resize, in multitasking mode. This is required to prevent us from zooming
in excessively when the width of the view increases, since we'd otherwise attempt to keep the same content in
the page visible by zooming in (for instance, if an image covers most of the visual viewport at a lower view
width, this `min()` logic would cause us to zoom in, such that the image would still cover most of the viewport
at a larger width). This behavior is undesirable in multitasking mode.

* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::creationParameters):
(WebKit::WebPageProxy::setIsInMultitaskingMode):
* UIProcess/WebPageProxy.h:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::didCommitLoad):
(WebKit::WebPage::setIsInMultitaskingMode):

Add plumbing to set the `m_isInMultitaskingMode` flag on WebPage, and update the viewport configuration flag to
prefer horizontal scrolling below 980pt.

* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::scaleAfterViewportWidthChange):

Refactor a bit of code here (without changing any behaviors), to make it a bit clearer:

-   Rename `userHasChangedPageScaleFactor` to `scaleToFitContent` to better describe how this flag affects the
    adjusted target scale during dynamic resize.

-   Make the log messages specific to both branches, and also log the adjusted viewport scale instead of the
    (currently unused) given `scale` in the non-`scaleToFitContent` codepath.

(WebKit::WebPage::dynamicViewportSizeUpdate):

Make another "multitasking mode viewport behavior" adjustment here by maintaining the initial scale (only if the
viewport was already at initial scale) when performing dynamic viewport size updates. By default, we currently
adjust the scale such that the same content is still visible at the new viewport size; however, when allowing
horizontal scrolling, this causes us to zoom in excessively when making the window width larger. Instead, when
multitasking mode is enabled, we should try to preserve initial scale when changing window size, such that only
the horizontal scroll amount changes.

(WebKit::WebPage::usesMultitaskingModeViewportBehaviors const):

Add a helper method to encapsulate whether or not multitasking mode viewport behaviors should be used; this
should be true only when both desktop-class viewport behaviors are active, *and* multitasking mode is also
active.

Canonical link: https://commits.webkit.org/250431@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@293994 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebCore/ChangeLog

@@ -1,3 +1,26 @@
+2022-05-07  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS] Adjust some viewport behaviors when multitasking mode is enabled
+        https://bugs.webkit.org/show_bug.cgi?id=240151
+        rdar://87157773
+
+        Reviewed by Tim Horton.
+
+        Add a new ViewportConfiguration flag to prefer horizontal scrolling over shrinking to fit when the view layout
+        size falls under the current "default desktop webpage" of 980pt. See WebKit changes for more details.
+
+        * page/ViewportConfiguration.cpp:
+        (WebCore::ViewportConfiguration::initialScaleFromSize const):
+        (WebCore::ViewportConfiguration::webpageParameters):
+        (WebCore::ViewportConfiguration::imageDocumentParameters):
+
+        Pull the magic value representing the "assumed width of most desktop webpages" (980) out into a named constant,
+        `defaultDesktopViewportWidth`, so that we can consult it when computing the initial scale.
+
+        (WebCore::ViewportConfiguration::description const):
+        * page/ViewportConfiguration.h:
+        (WebCore::ViewportConfiguration::setPrefersHorizontalScrollingBelowDesktopViewportWidths):
+
 2022-05-09  Tim Horton  <timothy_horton@apple.com>
 
         Const-ify Node::willRespondTo*Events()

Source/WebCore/page/ViewportConfiguration.cpp

@@ -38,6 +38,8 @@
 
 namespace WebCore {
 
+constexpr double defaultDesktopViewportWidth = 980;
+
 #if ASSERT_ENABLED
 static bool constraintsAreAllRelative(const ViewportConfiguration::Parameters& configuration)
 {
@@ -281,8 +283,12 @@ double ViewportConfiguration::initialScaleFromSize(double width, double height,
             initialScale = m_viewLayoutSize.width() / m_configuration.width;
         else if (shouldShrinkToFitMinimumEffectiveDeviceWidthWhenIgnoringScalingConstraints())
             initialScale = effectiveLayoutSizeScaleFactor();
-        else if (width > 0)
-            initialScale = m_viewLayoutSize.width() / width;
+        else if (width > 0) {
+            auto shrinkToFitWidth = m_viewLayoutSize.width();
+            if (m_prefersHorizontalScrollingBelowDesktopViewportWidths)
+                shrinkToFitWidth = std::max<float>(shrinkToFitWidth, std::min(width, defaultDesktopViewportWidth));
+            initialScale = shrinkToFitWidth / width;
+        }
     }
 
     // Prevent the initial scale from shrinking to a height smaller than our view's minimum height.
@@ -389,7 +395,7 @@ ViewportConfiguration::Parameters ViewportConfiguration::nativeWebpageParameters
 ViewportConfiguration::Parameters ViewportConfiguration::webpageParameters()
 {
     Parameters parameters;
-    parameters.width = 980;
+    parameters.width = defaultDesktopViewportWidth;
     parameters.widthIsSet = true;
     parameters.allowsUserScaling = true;
     parameters.allowsShrinkToFit = true;
@@ -420,7 +426,7 @@ ViewportConfiguration::Parameters ViewportConfiguration::textDocumentParameters(
 ViewportConfiguration::Parameters ViewportConfiguration::imageDocumentParameters()
 {
     Parameters parameters;
-    parameters.width = 980;
+    parameters.width = defaultDesktopViewportWidth;
     parameters.widthIsSet = true;
     parameters.allowsUserScaling = true;
     parameters.allowsShrinkToFit = false;
@@ -724,6 +730,7 @@ String ViewportConfiguration::description() const
     ts.dumpProperty("minimum effective device width (for view)", m_minimumEffectiveDeviceWidthForView);
     ts.dumpProperty("minimum effective device width (for shrink-to-fit)", m_minimumEffectiveDeviceWidthForShrinkToFit);
     ts.dumpProperty("known to lay out wider than viewport", m_isKnownToLayOutWiderThanViewport ? "true" : "false");
+    ts.dumpProperty("prefers horizontal scrolling", m_prefersHorizontalScrollingBelowDesktopViewportWidths ? "true" : "false");
 
     ts.endGroup();
 

Source/WebCore/page/ViewportConfiguration.h

@@ -128,6 +128,8 @@ class ViewportConfiguration {
     void setForceAlwaysUserScalable(bool forceAlwaysUserScalable) { m_forceAlwaysUserScalable = forceAlwaysUserScalable; }
     double layoutSizeScaleFactor() const { return m_layoutSizeScaleFactor; }
 
+    void setPrefersHorizontalScrollingBelowDesktopViewportWidths(bool value) { m_prefersHorizontalScrollingBelowDesktopViewportWidths = value; }
+
     WEBCORE_EXPORT IntSize layoutSize() const;
     WEBCORE_EXPORT int layoutWidth() const;
     WEBCORE_EXPORT int layoutHeight() const;
@@ -209,6 +211,7 @@ class ViewportConfiguration {
     bool m_canIgnoreScalingConstraints;
     bool m_forceAlwaysUserScalable;
     bool m_isKnownToLayOutWiderThanViewport { false };
+    bool m_prefersHorizontalScrollingBelowDesktopViewportWidths { false };
 };
 
 WTF::TextStream& operator<<(WTF::TextStream&, const ViewportConfiguration::Parameters&);

Source/WebKit/ChangeLog

@@ -1,3 +1,78 @@
+2022-05-07  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        [iOS] Adjust some viewport behaviors when multitasking mode is enabled
+        https://bugs.webkit.org/show_bug.cgi?id=240151
+        rdar://87157773
+
+        Reviewed by Tim Horton.
+
+        Make some adjustments to viewport behaviors when multitasking mode is enabled. See the comments below for more
+        details. There are no changes in behavior when multitasking mode is disabled; tests for the new behaviors in
+        multitasking mode will be added in a subsequent patch.
+
+        * Shared/WebPageCreationParameters.cpp:
+        (WebKit::WebPageCreationParameters::encode const):
+        (WebKit::WebPageCreationParameters::decode):
+        * Shared/WebPageCreationParameters.h:
+
+        Add plumbing to inform the web process when "multitasking mode" state changes; we use this bit in WebPage to
+        determine whether or not we should use multitasking mode viewport behaviors (see below).
+
+        * UIProcess/API/ios/WKWebViewIOS.mm:
+        (-[WKWebView _registerForNotifications]):
+        (-[WKWebView didMoveToWindow]):
+        (-[WKWebView _multitaskingModeDidChange:]):
+
+        Send IPC to WebPage in these two places, to keep "multitasking mode" state in sync with the native view.
+
+        (-[WKWebView _beginAnimatedResizeWithUpdates:]):
+
+        Make a minor adjustment here to ignore `oldWebViewWidthInContentViewCoordinates` when computing a target scale
+        to zoom to when performing animated resize, in multitasking mode. This is required to prevent us from zooming
+        in excessively when the width of the view increases, since we'd otherwise attempt to keep the same content in
+        the page visible by zooming in (for instance, if an image covers most of the visual viewport at a lower view
+        width, this `min()` logic would cause us to zoom in, such that the image would still cover most of the viewport
+        at a larger width). This behavior is undesirable in multitasking mode.
+
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::creationParameters):
+        (WebKit::WebPageProxy::setIsInMultitaskingMode):
+        * UIProcess/WebPageProxy.h:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::didCommitLoad):
+        (WebKit::WebPage::setIsInMultitaskingMode):
+
+        Add plumbing to set the `m_isInMultitaskingMode` flag on WebPage, and update the viewport configuration flag to
+        prefer horizontal scrolling below 980pt.
+
+        * WebProcess/WebPage/WebPage.h:
+        * WebProcess/WebPage/WebPage.messages.in:
+        * WebProcess/WebPage/ios/WebPageIOS.mm:
+        (WebKit::scaleAfterViewportWidthChange):
+
+        Refactor a bit of code here (without changing any behaviors), to make it a bit clearer:
+
+        -   Rename `userHasChangedPageScaleFactor` to `scaleToFitContent` to better describe how this flag affects the
+            adjusted target scale during dynamic resize.
+
+        -   Make the log messages specific to both branches, and also log the adjusted viewport scale instead of the
+            (currently unused) given `scale` in the non-`scaleToFitContent` codepath.
+
+        (WebKit::WebPage::dynamicViewportSizeUpdate):
+
+        Make another "multitasking mode viewport behavior" adjustment here by maintaining the initial scale (only if the
+        viewport was already at initial scale) when performing dynamic viewport size updates. By default, we currently
+        adjust the scale such that the same content is still visible at the new viewport size; however, when allowing
+        horizontal scrolling, this causes us to zoom in excessively when making the window width larger. Instead, when
+        multitasking mode is enabled, we should try to preserve initial scale when changing window size, such that only
+        the horizontal scroll amount changes.
+
+        (WebKit::WebPage::usesMultitaskingModeViewportBehaviors const):
+
+        Add a helper method to encapsulate whether or not multitasking mode viewport behaviors should be used; this
+        should be true only when both desktop-class viewport behaviors are active, *and* multitasking mode is also
+        active.
+
 2022-05-09  Ben Nham  <nham@apple.com>
 
         Allow log streaming from webpushd

Source/WebKit/Shared/WebPageCreationParameters.cpp

@@ -190,6 +190,10 @@ void WebPageCreationParameters::encode(IPC::Encoder& encoder) const
     encoder << requiresUserActionForEditingControlsManager;
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+    encoder << isInMultitaskingMode;
+#endif
+
     encoder << contentSecurityPolicyModeForExtension;
 }
 
@@ -601,6 +605,11 @@ std::optional<WebPageCreationParameters> WebPageCreationParameters::decode(IPC::
         return std::nullopt;
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+    if (!decoder.decode(parameters.isInMultitaskingMode))
+        return std::nullopt;
+#endif
+
     if (!decoder.decode(parameters.contentSecurityPolicyModeForExtension))
         return std::nullopt;
 

Source/WebKit/Shared/WebPageCreationParameters.h

@@ -266,6 +266,8 @@ struct WebPageCreationParameters {
     bool requiresUserActionForEditingControlsManager { false };
 #endif
 
+    bool isInMultitaskingMode { false };
+
     WebCore::ContentSecurityPolicyModeForExtension contentSecurityPolicyModeForExtension { WebCore::ContentSecurityPolicyModeForExtension::None };
 };
 

Source/WebKit/UIProcess/API/ios/WKWebViewIOS.mm

@@ -195,6 +195,10 @@ - (void)_registerForNotifications
     [center addObserver:self selector:@selector(_accessibilitySettingsDidChange:) name:UIAccessibilityGrayscaleStatusDidChangeNotification object:nil];
     [center addObserver:self selector:@selector(_accessibilitySettingsDidChange:) name:UIAccessibilityInvertColorsStatusDidChangeNotification object:nil];
     [center addObserver:self selector:@selector(_accessibilitySettingsDidChange:) name:UIAccessibilityReduceMotionStatusDidChangeNotification object:nil];
+
+#if HAVE(MULTITASKING_MODE)
+    [center addObserver:self selector:@selector(_multitaskingModeDidChange:) name:self.multitaskingModeChangedNotificationName object:nil];
+#endif
 }
 
 - (BOOL)_isShowingVideoPictureInPicture
@@ -1554,6 +1558,10 @@ - (void)didMoveToWindow
     _page->activityStateDidChange(WebCore::ActivityState::allFlags());
     _page->webViewDidMoveToWindow();
     [self _presentCaptivePortalModeAlertIfNeeded];
+#if HAVE(MULTITASKING_MODE)
+    if (_page->hasRunningProcess() && self.window)
+        _page->setIsInMultitaskingMode(self._isInMultitaskingMode);
+#endif
 }
 
 - (void)_setOpaqueInternal:(BOOL)opaque
@@ -2723,6 +2731,21 @@ - (void)buildMenuWithBuilder:(id <UIMenuBuilder>)builder
 
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+
+- (void)_multitaskingModeDidChange:(NSNotification *)notification
+{
+    if (dynamic_objc_cast<UIWindowScene>(notification.object) != self.window.windowScene)
+        return;
+
+    if (!_page || !_page->hasRunningProcess())
+        return;
+
+    _page->setIsInMultitaskingMode(self._isInMultitaskingMode);
+}
+
+#endif // HAVE(MULTITASKING_MODE)
+
 @end
 
 @implementation WKWebView (WKPrivateIOS)
@@ -3222,7 +3245,13 @@ - (void)_beginAnimatedResizeWithUpdates:(void (^)(void))updateBlock
 
     // Compute the new scale to keep the current content width in the scrollview.
     CGFloat oldWebViewWidthInContentViewCoordinates = oldUnobscuredContentRect.width();
-    _animatedResizeOriginalContentWidth = std::min(contentSizeInContentViewCoordinates.width, oldWebViewWidthInContentViewCoordinates);
+    _animatedResizeOriginalContentWidth = [&] {
+#if HAVE(MULTITASKING_MODE)
+        if (self._isInMultitaskingMode)
+            return contentSizeInContentViewCoordinates.width;
+#endif
+        return std::min(contentSizeInContentViewCoordinates.width, oldWebViewWidthInContentViewCoordinates);
+    }();
     CGFloat targetScale = newViewLayoutSize.width() / _animatedResizeOriginalContentWidth;
     CGFloat resizeAnimationViewAnimationScale = targetScale / contentZoomScale(self);
     [_resizeAnimationView setTransform:CGAffineTransformMakeScale(resizeAnimationViewAnimationScale, resizeAnimationViewAnimationScale)];

Source/WebKit/UIProcess/WebPageProxy.cpp

@@ -8514,6 +8514,10 @@ WebPageCreationParameters WebPageProxy::creationParameters(WebProcessProxy& proc
     parameters.requiresUserActionForEditingControlsManager = m_configuration->requiresUserActionForEditingControlsManager();
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+    parameters.isInMultitaskingMode = pageClient().isInMultitaskingMode();
+#endif
+
     return parameters;
 }
 
@@ -11391,6 +11395,15 @@ void WebPageProxy::shouldAllowImageMarkup(const ElementContext& context, Complet
 
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+
+void WebPageProxy::setIsInMultitaskingMode(bool isInMultitaskingMode)
+{
+    send(Messages::WebPage::SetIsInMultitaskingMode(isInMultitaskingMode));
+}
+
+#endif
+
 } // namespace WebKit
 
 #undef WEBPAGEPROXY_RELEASE_LOG

Source/WebKit/UIProcess/WebPageProxy.h

@@ -2105,6 +2105,10 @@ class WebPageProxy final : public API::ObjectImpl<API::Object::Type::Page>
     void shouldAllowImageMarkup(const WebCore::ElementContext&, CompletionHandler<void(bool)>&&);
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+    void setIsInMultitaskingMode(bool);
+#endif
+
 private:
     WebPageProxy(PageClient&, WebProcessProxy&, Ref<API::PageConfiguration>&&);
     void platformInitialize();

Source/WebKit/WebProcess/WebPage/WebPage.cpp

@@ -542,6 +542,9 @@ WebPage::WebPage(PageIdentifier pageID, WebPageCreationParameters&& parameters)
 #if HAVE(TOUCH_BAR)
     , m_requiresUserActionForEditingControlsManager(parameters.requiresUserActionForEditingControlsManager)
 #endif
+#if HAVE(MULTITASKING_MODE)
+    , m_isInMultitaskingMode(parameters.isInMultitaskingMode)
+#endif
 #if ENABLE(META_VIEWPORT)
     , m_forceAlwaysUserScalable(parameters.ignoresViewportScaleLimits)
 #endif
@@ -6702,6 +6705,8 @@ void WebPage::didCommitLoad(WebFrame* frame)
 
     bool viewportChanged = false;
 
+    m_viewportConfiguration.setPrefersHorizontalScrollingBelowDesktopViewportWidths(usesMultitaskingModeViewportBehaviors());
+
     LOG_WITH_STREAM(VisibleRects, stream << "WebPage " << m_identifier.toUInt64() << " didCommitLoad setting content size to " << coreFrame->view()->contentsSize());
     if (m_viewportConfiguration.setContentsSize(coreFrame->view()->contentsSize()))
         viewportChanged = true;
@@ -8112,6 +8117,19 @@ void WebPage::shouldAllowImageMarkup(const ElementContext& context, CompletionHa
 
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+
+void WebPage::setIsInMultitaskingMode(bool value)
+{
+    if (m_isInMultitaskingMode == value)
+        return;
+
+    m_isInMultitaskingMode = value;
+    m_viewportConfiguration.setPrefersHorizontalScrollingBelowDesktopViewportWidths(usesMultitaskingModeViewportBehaviors());
+}
+
+#endif // HAVE(MULTITASKING_MODE)
+
 } // namespace WebKit
 
 #undef WEBPAGE_RELEASE_LOG

Source/WebKit/WebProcess/WebPage/WebPage.h

@@ -1542,6 +1542,10 @@ class WebPage : public API::ObjectImpl<API::Object::Type::BundlePage>, public IP
     void shouldAllowImageMarkup(const WebCore::ElementContext&, CompletionHandler<void(bool)>&&) const;
 #endif
 
+#if HAVE(MULTITASKING_MODE)
+    void setIsInMultitaskingMode(bool);
+#endif
+
 private:
     WebPage(WebCore::PageIdentifier, WebPageCreationParameters&&);
 
@@ -1791,6 +1795,10 @@ class WebPage : public API::ObjectImpl<API::Object::Type::BundlePage>, public IP
 
     void endPrintingImmediately();
 
+#if ENABLE(META_VIEWPORT)
+    bool usesMultitaskingModeViewportBehaviors() const;
+#endif
+
 #if HAVE(APP_ACCENT_COLORS)
     void setAccentColor(WebCore::Color);
 #endif
@@ -2258,6 +2266,10 @@ class WebPage : public API::ObjectImpl<API::Object::Type::BundlePage>, public IP
 #endif
     OptionSet<WebCore::ActivityState::Flag> m_lastActivityStateChanges;
 
+#if HAVE(MULTITASKING_MODE)
+    bool m_isInMultitaskingMode { false };
+#endif
+
 #if ENABLE(CONTEXT_MENUS)
     bool m_waitingForContextMenuToShow { false };
 #endif

Source/WebKit/WebProcess/WebPage/WebPage.messages.in

@@ -688,4 +688,8 @@ GenerateSyntheticEditingCommand(enum:uint8_t WebKit::SyntheticEditingCommandType
     ModelInlinePreviewDidLoad(WebCore::GraphicsLayer::PlatformLayerID layerID)
     ModelInlinePreviewDidFailToLoad(WebCore::GraphicsLayer::PlatformLayerID layerID, WebCore::ResourceError error)
 #endif
+
+#if HAVE(MULTITASKING_MODE)
+    SetIsInMultitaskingMode(bool isInMultitaskingMode)
+#endif
 }