Promote WKWebView media playback SPI to API

https://bugs.webkit.org/show_bug.cgi?id=217335
<rdar://problem/63406100>

Reviewed by Jer Noble.

Source/WebCore:

Change stopAllMediaPlayback to be more aptly named
pauseAllMediaPlayback because it is possible to resume
playing media after this is called. 'Stop' implies more permanence.
Add a function to the Document and Page objects to see if media playback
exists, is paused, or is suspended to determine media playback state.

* dom/Document.cpp:
(WebCore::Document::~Document):
(WebCore::Document::mediaPlaybackExists):
(WebCore::Document::mediaPlaybackIsPaused):
(WebCore::Document::pauseAllMediaPlayback):
(WebCore::Document::stopAllMediaPlayback): Deleted.
* dom/Document.h:
* page/Page.cpp:
(WebCore::Page::mediaPlaybackExists):
(WebCore::Page::mediaPlaybackIsPaused):
(WebCore::Page::pauseAllMediaPlayback):
(WebCore::Page::stopAllMediaPlayback): Deleted.
* page/Page.h:
* platform/audio/PlatformMediaSessionManager.cpp:
(WebCore::PlatformMediaSessionManager::pauseAllMediaPlaybackForDocument):
(WebCore::PlatformMediaSessionManager::mediaPlaybackIsPaused):
(WebCore::PlatformMediaSessionManager::stopAllMediaPlaybackForDocument): Deleted.
* platform/audio/PlatformMediaSessionManager.h:

Source/WebKit:

Currently the only way to interact with video in WKWebView is by injecting
JavaScript into a page. It would be useful to promote various media
playback SPI to API to make this easier for clients, including adding
completion handlers to the new API calls. Along with this,
we should add new API to check for media playback state.
This patch keeps the SPI around for compatibility.

* Shared/API/Cocoa/WebKit.h:
* Shared/MediaPlaybackState.h: Added.
* UIProcess/API/Cocoa/WKMediaPlaybackState.h: Added.
* UIProcess/API/Cocoa/WKWebView.h:
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView closeAllMediaPresentations]):
(-[WKWebView pauseAllMediaPlayback:]):
(-[WKWebView suspendAllMediaPlayback:]):
(-[WKWebView resumeAllMediaPlayback:]):
(toWKMediaPlaybackState):
(-[WKWebView requestMediaPlaybackState:]):
(-[WKWebView _closeAllMediaPresentations]):
(-[WKWebView _stopAllMediaPlayback]):
(-[WKWebView _suspendAllMediaPlayback]):
(-[WKWebView _resumeAllMediaPlayback]):
* UIProcess/ios/fullscreen/WKFullScreenViewController.mm:
(-[WKFullScreenViewController _showPhishingAlert]):
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::requestMediaPlaybackState):
(WebKit::WebPageProxy::pauseAllMediaPlayback):
(WebKit::WebPageProxy::suspendAllMediaPlayback):
(WebKit::WebPageProxy::resumeAllMediaPlayback):
(WebKit::WebPageProxy::stopAllMediaPlayback): Deleted.
* UIProcess/WebPageProxy.h:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::requestMediaPlaybackState):
(WebKit::WebPage::pauseAllMediaPlayback):
(WebKit::WebPage::suspendAllMediaPlayback):
(WebKit::WebPage::resumeAllMediaPlayback):
(WebKit::WebPage::stopAllMediaPlayback): Deleted.
* WebProcess/WebPage/WebPage.h:
* WebProcess/WebPage/WebPage.messages.in:

Tools:

Add API testing coverage for new API. Also converted testing to
use the new API instead of SPI.

* TestWebKitAPI/Tests/WebKitCocoa/StopSuspendResumeAllMedia.mm:
(TestWebKitAPI::TEST):


Canonical link: https://commits.webkit.org/230299@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@268268 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebCore/ChangeLog

@@ -1,3 +1,36 @@
+2020-10-09  Kate Cheney  <katherine_cheney@apple.com>
+
+        Promote WKWebView media playback SPI to API
+        https://bugs.webkit.org/show_bug.cgi?id=217335
+        <rdar://problem/63406100>
+
+        Reviewed by Jer Noble.
+
+        Change stopAllMediaPlayback to be more aptly named
+        pauseAllMediaPlayback because it is possible to resume
+        playing media after this is called. 'Stop' implies more permanence.
+        Add a function to the Document and Page objects to see if media playback
+        exists, is paused, or is suspended to determine media playback state.
+
+        * dom/Document.cpp:
+        (WebCore::Document::~Document):
+        (WebCore::Document::mediaPlaybackExists):
+        (WebCore::Document::mediaPlaybackIsPaused):
+        (WebCore::Document::pauseAllMediaPlayback):
+        (WebCore::Document::stopAllMediaPlayback): Deleted.
+        * dom/Document.h:
+        * page/Page.cpp:
+        (WebCore::Page::mediaPlaybackExists):
+        (WebCore::Page::mediaPlaybackIsPaused):
+        (WebCore::Page::pauseAllMediaPlayback):
+        (WebCore::Page::stopAllMediaPlayback): Deleted.
+        * page/Page.h:
+        * platform/audio/PlatformMediaSessionManager.cpp:
+        (WebCore::PlatformMediaSessionManager::pauseAllMediaPlaybackForDocument):
+        (WebCore::PlatformMediaSessionManager::mediaPlaybackIsPaused):
+        (WebCore::PlatformMediaSessionManager::stopAllMediaPlaybackForDocument): Deleted.
+        * platform/audio/PlatformMediaSessionManager.h:
+
 2020-10-09  Peng Liu  <peng.liu6@apple.com>
 
         Move some audio buffer management related files from folder "mac" to folder "cocoa"

Source/WebCore/dom/Document.cpp

@@ -702,7 +702,7 @@ Document::~Document()
         m_cachedResourceLoader->setDocument(nullptr);
 
 #if ENABLE(VIDEO)
-    stopAllMediaPlayback();
+    pauseAllMediaPlayback();
 #endif
 
     // We must call clearRareData() here since a Document class inherits TreeScope
@@ -1796,10 +1796,24 @@ void Document::forEachMediaElement(const Function<void(HTMLMediaElement&)>& func
         function(element);
 }
 
-void Document::stopAllMediaPlayback()
+bool Document::mediaPlaybackExists()
 {
     if (auto* platformMediaSessionManager = PlatformMediaSessionManager::sharedManagerIfExists())
-        platformMediaSessionManager->stopAllMediaPlaybackForDocument(identifier());
+        return !platformMediaSessionManager->hasNoSession();
+    return false;
+}
+
+bool Document::mediaPlaybackIsPaused()
+{
+    if (auto* platformMediaSessionManager = PlatformMediaSessionManager::sharedManagerIfExists())
+        return platformMediaSessionManager->mediaPlaybackIsPaused(identifier());
+    return false;
+}
+
+void Document::pauseAllMediaPlayback()
+{
+    if (auto* platformMediaSessionManager = PlatformMediaSessionManager::sharedManagerIfExists())
+        platformMediaSessionManager->pauseAllMediaPlaybackForDocument(identifier());
 }
 
 void Document::suspendAllMediaPlayback()

Source/WebCore/dom/Document.h

@@ -1113,7 +1113,9 @@ class Document
     void unregisterForVisibilityStateChangedCallbacks(VisibilityChangeClient&);
 
 #if ENABLE(VIDEO)
-    void stopAllMediaPlayback();
+    bool mediaPlaybackExists();
+    bool mediaPlaybackIsPaused();
+    void pauseAllMediaPlayback();
     void suspendAllMediaPlayback();
     void resumeAllMediaPlayback();
     void suspendAllMediaBuffering();

Source/WebCore/page/Page.cpp

@@ -2066,11 +2066,39 @@ void Page::stopMediaCapture()
 #endif
 }
 
-void Page::stopAllMediaPlayback()
+bool Page::mediaPlaybackExists()
+{
+#if ENABLE(VIDEO)
+    bool mediaPlaybackExists = false;
+    forEachDocument([&mediaPlaybackExists] (Document& document) {
+        if (document.mediaPlaybackExists())
+            mediaPlaybackExists = true;
+    });
+    return mediaPlaybackExists;
+#else
+    return false;
+#endif
+}
+
+bool Page::mediaPlaybackIsPaused()
+{
+#if ENABLE(VIDEO)
+    bool mediaPlaybackIsPaused = false;
+    forEachDocument([&mediaPlaybackIsPaused] (Document& document) {
+        if (document.mediaPlaybackIsPaused())
+            mediaPlaybackIsPaused = true;
+    });
+    return mediaPlaybackIsPaused;
+#else
+    return false;
+#endif
+}
+
+void Page::pauseAllMediaPlayback()
 {
 #if ENABLE(VIDEO)
     forEachDocument([] (Document& document) {
-        document.stopAllMediaPlayback();
+        document.pauseAllMediaPlayback();
     });
 #endif
 }

Source/WebCore/page/Page.h

@@ -694,7 +694,9 @@ class Page : public Supplementable<Page>, public CanMakeWeakPtr<Page> {
     WEBCORE_EXPORT void setMuted(MediaProducer::MutedStateFlags);
     WEBCORE_EXPORT void stopMediaCapture();
 
-    WEBCORE_EXPORT void stopAllMediaPlayback();
+    WEBCORE_EXPORT bool mediaPlaybackExists();
+    WEBCORE_EXPORT bool mediaPlaybackIsPaused();
+    WEBCORE_EXPORT void pauseAllMediaPlayback();
     WEBCORE_EXPORT void suspendAllMediaPlayback();
     WEBCORE_EXPORT void resumeAllMediaPlayback();
     bool mediaPlaybackIsSuspended() const { return m_mediaPlaybackIsSuspended; }

Source/WebCore/platform/audio/PlatformMediaSessionManager.cpp

@@ -459,13 +459,24 @@ void PlatformMediaSessionManager::processSystemDidWake()
     });
 }
 
-void PlatformMediaSessionManager::stopAllMediaPlaybackForDocument(DocumentIdentifier documentIdentifier)
+void PlatformMediaSessionManager::pauseAllMediaPlaybackForDocument(DocumentIdentifier documentIdentifier)
 {
     forEachDocumentSession(documentIdentifier, [](auto& session) {
         session.pauseSession();
     });
 }
 
+
+bool PlatformMediaSessionManager::mediaPlaybackIsPaused(DocumentIdentifier documentIdentifier)
+{
+    bool mediaPlaybackIsPaused = false;
+    forEachDocumentSession(documentIdentifier, [&mediaPlaybackIsPaused](auto& session) {
+        if (session.state() == PlatformMediaSession::Paused)
+            mediaPlaybackIsPaused = true;
+    });
+    return mediaPlaybackIsPaused;
+}
+
 void PlatformMediaSessionManager::stopAllMediaPlaybackForProcess()
 {
     forEachSession([] (auto& session) {

Source/WebCore/platform/audio/PlatformMediaSessionManager.h

@@ -88,7 +88,8 @@ class PlatformMediaSessionManager
     WEBCORE_EXPORT void processWillSuspend();
     WEBCORE_EXPORT void processDidResume();
 
-    void stopAllMediaPlaybackForDocument(DocumentIdentifier);
+    bool mediaPlaybackIsPaused(DocumentIdentifier);
+    void pauseAllMediaPlaybackForDocument(DocumentIdentifier);
     WEBCORE_EXPORT void stopAllMediaPlaybackForProcess();
 
     void suspendAllMediaPlaybackForDocument(DocumentIdentifier);
@@ -143,6 +144,7 @@ class PlatformMediaSessionManager
     WEBCORE_EXPORT void processDidReceiveRemoteControlCommand(PlatformMediaSession::RemoteControlCommandType, const PlatformMediaSession::RemoteCommandArgument*);
 
     bool isInterrupted() const { return m_interrupted; }
+    bool hasNoSession() const;
 
 protected:
     friend class PlatformMediaSession;
@@ -168,7 +170,6 @@ class PlatformMediaSessionManager
 #endif
 
     int countActiveAudioCaptureSources();
-    bool hasNoSession() const;
 
     bool computeSupportsSeeking() const;
 

Source/WebKit/ChangeLog

@@ -1,3 +1,52 @@
+2020-10-09  Kate Cheney  <katherine_cheney@apple.com>
+
+        Promote WKWebView media playback SPI to API
+        https://bugs.webkit.org/show_bug.cgi?id=217335
+        <rdar://problem/63406100>
+
+        Reviewed by Jer Noble.
+
+        Currently the only way to interact with video in WKWebView is by injecting
+        JavaScript into a page. It would be useful to promote various media
+        playback SPI to API to make this easier for clients, including adding
+        completion handlers to the new API calls. Along with this,
+        we should add new API to check for media playback state.
+        This patch keeps the SPI around for compatibility.
+
+        * Shared/API/Cocoa/WebKit.h:
+        * Shared/MediaPlaybackState.h: Added.
+        * UIProcess/API/Cocoa/WKMediaPlaybackState.h: Added.
+        * UIProcess/API/Cocoa/WKWebView.h:
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView closeAllMediaPresentations]):
+        (-[WKWebView pauseAllMediaPlayback:]):
+        (-[WKWebView suspendAllMediaPlayback:]):
+        (-[WKWebView resumeAllMediaPlayback:]):
+        (toWKMediaPlaybackState):
+        (-[WKWebView requestMediaPlaybackState:]):
+        (-[WKWebView _closeAllMediaPresentations]):
+        (-[WKWebView _stopAllMediaPlayback]):
+        (-[WKWebView _suspendAllMediaPlayback]):
+        (-[WKWebView _resumeAllMediaPlayback]):
+        * UIProcess/ios/fullscreen/WKFullScreenViewController.mm:
+        (-[WKFullScreenViewController _showPhishingAlert]):
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::requestMediaPlaybackState):
+        (WebKit::WebPageProxy::pauseAllMediaPlayback):
+        (WebKit::WebPageProxy::suspendAllMediaPlayback):
+        (WebKit::WebPageProxy::resumeAllMediaPlayback):
+        (WebKit::WebPageProxy::stopAllMediaPlayback): Deleted.
+        * UIProcess/WebPageProxy.h:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::requestMediaPlaybackState):
+        (WebKit::WebPage::pauseAllMediaPlayback):
+        (WebKit::WebPage::suspendAllMediaPlayback):
+        (WebKit::WebPage::resumeAllMediaPlayback):
+        (WebKit::WebPage::stopAllMediaPlayback): Deleted.
+        * WebProcess/WebPage/WebPage.h:
+        * WebProcess/WebPage/WebPage.messages.in:
+
 2020-10-09  Alex Christensen  <achristensen@webkit.org>
 
         Use sendWithAsyncReply for NetworkProcess::CancelDownload

Source/WebKit/Shared/API/Cocoa/WebKit.h

@@ -36,6 +36,7 @@
 #import <WebKit/WKFoundation.h>
 #import <WebKit/WKFrameInfo.h>
 #import <WebKit/WKHTTPCookieStore.h>
+#import <WebKit/WKMediaPlaybackState.h>
 #import <WebKit/WKNavigation.h>
 #import <WebKit/WKNavigationAction.h>
 #import <WebKit/WKNavigationDelegate.h>

Source/WebKit/Shared/MediaPlaybackState.h

@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace WebKit {
+
+enum class MediaPlaybackState : uint8_t {
+    NoMediaPlayback,
+    MediaPlaybackPaused,
+    MediaPlaybackSuspended,
+    MediaPlaybackPlaying
+};
+
+} // namespace WebKit
+
+namespace WTF {
+
+template<> struct EnumTraits<WebKit::MediaPlaybackState> {
+    using values = EnumValues<
+        WebKit::MediaPlaybackState,
+        WebKit::MediaPlaybackState::NoMediaPlayback,
+        WebKit::MediaPlaybackState::MediaPlaybackPaused,
+        WebKit::MediaPlaybackState::MediaPlaybackSuspended,
+        WebKit::MediaPlaybackState::MediaPlaybackPlaying
+    >;
+};
+
+} // namespace WTF

Source/WebKit/UIProcess/API/Cocoa/WKMediaPlaybackState.h

@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#import <Foundation/Foundation.h>
+#import <WebKit/WKFoundation.h>
+
+NS_ASSUME_NONNULL_BEGIN
+
+typedef NS_ENUM(NSUInteger, WKMediaPlaybackState) {
+    WKMediaPlaybackStateNone,
+    WKMediaPlaybackStatePaused,
+    WKMediaPlaybackStateSuspended,
+    WKMediaPlaybackStatePlaying
+} WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
+NS_ASSUME_NONNULL_END

Source/WebKit/UIProcess/API/Cocoa/WKWebView.h

@@ -25,6 +25,8 @@
 
 #import <WebKit/WKFoundation.h>
 
+#import <WebKit/WKMediaPlaybackState.h>
+
 #if TARGET_OS_IPHONE
 #import <UIKit/UIKit.h>
 #else
@@ -321,6 +323,35 @@ WK_CLASS_AVAILABLE(macos(10.10), ios(8.0))
 */
 - (void)callAsyncJavaScript:(NSString *)functionBody arguments:(nullable NSDictionary<NSString *, id> *)arguments inFrame:(nullable WKFrameInfo *)frame inContentWorld:(WKContentWorld *)contentWorld completionHandler:(void (^ _Nullable)(_Nullable id, NSError * _Nullable error))completionHandler NS_REFINED_FOR_SWIFT WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
+/*! @abstract Closes all out-of-window media presentations in a WKWebView.
+ @discussion Includes picture-in-picture and fullscreen.
+ */
+- (void)closeAllMediaPresentations WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
+/*! @abstract Pauses media playback in WKWebView.
+ @discussion Pauses media playback. Media in the page can be restarted by calling play() on a media element or resume() on an AudioContext.
+ */
+- (void)pauseAllMediaPlayback:(void (^_Nullable)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
+/*! @abstract Suspends media playback in WKWebView.
+ @discussion Pauses media playback and blocks all attempts by the page to resume until resumeAllMediaPlayback is called. This should always be called in pairs with resumeAllMediaPlayback.
+ */
+- (void)suspendAllMediaPlayback:(void (^_Nullable)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
+/*! @abstract Resumes media playback in WKWebView.
+ @discussion This should always be called in pairs with suspendAllMediaPlayback.
+ */
+- (void)resumeAllMediaPlayback:(void (^ _Nullable)(void))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
+/*! @abstract Get the current media playback state of a WKWebView.
+ @param completionHandler A block to invoke with the return value of the function call.
+ @discussion If media playback exists, WKMediaPlaybackState will be one of three
+ values: WKMediaPlaybackPaused, WKMediaPlaybackSuspended, or WKMediaPlaybackPlaying.
+ If no media playback exists in the current WKWebView, WKMediaPlaybackState will equal
+ WKNoMediaPlayback.
+ */
+- (void)requestMediaPlaybackState:(void (^)(WKMediaPlaybackState))completionHandler WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
 /*! @abstract Get a snapshot for the visible viewport of WKWebView.
  @param snapshotConfiguration An object that specifies how the snapshot is configured.
  @param completionHandler A block to invoke when the snapshot is ready.