Mute capture when disconnected from hardware console #1691
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eric-carlson
force-pushed
the
mute-capture-when-disconnected-from-hardware-console
branch
2 times, most recently
from
e6d7d37 to
2a37553
Compare
…kend are mutated after copy https://bugs.webkit.org/show_bug.cgi?id=241367 Patch by Kimmo Kinnunen <kkinnunen@apple.com> on 2022-06-22 Reviewed by Said Abou-Hallawa. With GPUP, ImageBuffer IOSurface modifications happen in the GPUP. In r294536 it was changed that WP creates NativeImage instances with CGImage instances created from IOSurfaces mapped in WP. These CGImages do not know that the underlying IOSurface changes in the other process. Detach the CGImages from the IOSurface before the first write to the ImageBuffer is being sent to GPUP. This is done with the ensureNativeImagesHaveCopiedBackingStore() call. * LayoutTests/fast/canvas/canvas-pattern-from-modified-canvas-expected.txt: Added. * LayoutTests/fast/canvas/canvas-pattern-from-modified-canvas.html: Added. * Source/WebCore/platform/graphics/ImageBuffer.h: (WebCore::ImageBuffer::backingStoreWillChange): (WebCore::ImageBuffer::setNeedsFlush): Deleted. * Source/WebCore/platform/graphics/cg/ImageBufferIOSurfaceBackend.cpp: (WebCore::ImageBufferIOSurfaceBackend::~ImageBufferIOSurfaceBackend): (WebCore::ImageBufferIOSurfaceBackend::invalidateCachedNativeImage const): (WebCore::ImageBufferIOSurfaceBackend::copyNativeImage const): (WebCore::ImageBufferIOSurfaceBackend::ensureNativeImagesHaveCopiedBackingStore): * Source/WebCore/platform/graphics/cg/ImageBufferIOSurfaceBackend.h: * Source/WebKit/WebProcess/GPU/graphics/RemoteDisplayListRecorderProxy.h: (WebKit::RemoteDisplayListRecorderProxy::send): * Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h: (WebKit::RemoteImageBufferProxy::setNeedsFlush): (WebKit::RemoteImageBufferProxy::prepareForBackingStoreChange): Canonical link: https://commits.webkit.org/251751@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295746 268f45cc-cd09-0410-ab3c-d52691b4dbfc
https://bugs.webkit.org/show_bug.cgi?id=241820 <rdar://92834618 > Reviewed by Said Abou-Hallawa. from/to/values attributes should be stripped in presence of javascript URLs with the pref on. * Source/WebCore/dom/Element.cpp: (WebCore::Element::attributeContainsJavascriptURL const): (WebCore::Element::stripScriptingAttributes const): (WebCore::Element::isJavaScriptURLAttribute const): Deleted. * Source/WebCore/dom/Element.h: * Source/WebCore/editing/ReplaceSelectionCommand.cpp: (WebCore::ReplacementFragment::removeContentsWithSideEffects): * Source/WebCore/editing/markup.cpp: (WebCore::StyledMarkupAccumulator::appendStartTag): * Source/WebCore/svg/SVGAnimationElement.cpp: (WebCore::SVGAnimationElement::attributeContainsJavascriptURL const): * Source/WebCore/svg/SVGAnimationElement.h: * Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewConfiguration.mm: (TEST): Canonical link: https://commits.webkit.org/251752@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295747 268f45cc-cd09-0410-ab3c-d52691b4dbfc
brentfulgham
approved these changes
Jun 22, 2022
There was a problem hiding this comment.
Should this be Catalyst? I worry this won't compile without allowing it, but maybe it's not called anywhere in a Catalyst flow?
https://bugs.webkit.org/show_bug.cgi?id=241860 Reviewed by Alexey Proskuryakov. * Source/JavaScriptCore/Configurations/SDKVariant.xcconfig: * Source/ThirdParty/ANGLE/Configurations/SDKVariant.xcconfig: * Source/ThirdParty/gtest/xcode/Config/SDKVariant.xcconfig: * Source/ThirdParty/libwebrtc/Configurations/SDKVariant.xcconfig: * Source/WTF/Configurations/SDKVariant.xcconfig: * Source/WebCore/Configurations/SDKVariant.xcconfig: * Source/WebCore/PAL/Configurations/SDKVariant.xcconfig: * Source/WebGPU/Configurations/SDKVariant.xcconfig: * Source/WebInspectorUI/Configurations/SDKVariant.xcconfig: * Source/WebKit/Configurations/BaseTarget.xcconfig: * Source/WebKit/Configurations/SDKVariant.xcconfig: * Source/WebKitLegacy/mac/Configurations/SDKVariant.xcconfig: * Source/bmalloc/Configurations/SDKVariant.xcconfig: Canonical link: https://commits.webkit.org/251753@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295748 268f45cc-cd09-0410-ab3c-d52691b4dbfc
https://bugs.webkit.org/show_bug.cgi?id=240283 Reviewed by Carlos Garcia Campos. There are 4 cases that can happen after there has been a layerFlush and we're adopting the new state in the composition stage: 1. The layer is removed from the tree and the proxy is not assigned to any other layer: the deletion of the layer causes an invalidation of the proxy and both are destroyed afterwards. This works fine. 2. The layer is removed from the tree and the proxy is reassigned to a new layer: the deletion of the first layer causes the invalidation of the proxy, which is then activated on the second layer. As the first layer is destroyed, we don't have to worry about dangling references from it to the proxy's currentBuffer. This works fine. 3. The layer is kept in the tree and the proxy gets disassociated from it and not used by any other layer: we detect that the proxy is not used anymore and call invalidate on it, but the layer keeps a reference to the proxy's currentBuffer, which has been deleted during invalidate, which leads to a crash when trying to render the layer. 4. The layer is kept in the tree and the proxy gets associated to a new layer: as we detect that the proxy is still being used it's not invalidated, but it gets activated on the second layer. The first layer keeps a reference to the proxy's currentBuffer, which will be destroyed a bit later when swapBuffers is called on the proxy. This leads to a crash when trying to render the first layer. This patch addresses cases 3. and 4. described above. * Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxyGL.cpp: (WebCore::TextureMapperPlatformLayerProxyGL::activateOnCompositingThread): Ensure that the layer no longer keeps a reference to the current buffer if the proxy is already active on a different layer. (WebCore::TextureMapperPlatformLayerProxyGL::invalidate): Ensure that the invalidated layer does not keep a reference to the current buffer. Canonical link: https://commits.webkit.org/251754@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295749 268f45cc-cd09-0410-ab3c-d52691b4dbfc
https://bugs.webkit.org/show_bug.cgi?id=241872 Reviewed by Wenson Hsieh. with a capital S * Source/WebCore/dom/Element.cpp: (WebCore::Element::attributeContainsJavaScriptURL const): (WebCore::Element::stripScriptingAttributes const): (WebCore::Element::attributeContainsJavascriptURL const): Deleted. * Source/WebCore/dom/Element.h: * Source/WebCore/editing/ReplaceSelectionCommand.cpp: (WebCore::ReplacementFragment::removeContentsWithSideEffects): * Source/WebCore/editing/markup.cpp: (WebCore::StyledMarkupAccumulator::appendStartTag): * Source/WebCore/svg/SVGAnimationElement.cpp: (WebCore::SVGAnimationElement::attributeContainsJavaScriptURL const): (WebCore::SVGAnimationElement::attributeContainsJavascriptURL const): Deleted. * Source/WebCore/svg/SVGAnimationElement.h: Canonical link: https://commits.webkit.org/251755@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295750 268f45cc-cd09-0410-ab3c-d52691b4dbfc
…s SquareSpace, medium.com, and others) https://bugs.webkit.org/show_bug.cgi?id=241837 rdar://95658478 Reviewed by Aditya Keerthi. On iOS 16, the callout bar (which is now built on top of `UIEditMenuInteraction`) no longer respects evasion rects, passed in through the `UIWKInteractionViewProtocol` delegate method `-requestRectsToEvadeForSelectionCommandsWithCompletionHandler:`. This was previously used to avoid overlapping interactable controls on the page when presenting the callout bar, which the layout test `editing/selection/ios/avoid-showing-callout-menu-over-controls.html` exercises. To fix this, we're adding a replacement SPI in UIKit, allowing WebKit to vend a preferred `UIEditMenuArrowDirection` which will be consulted when presenting the edit menu interaction in order to show the callout bar. For more details, see: rdar://95652872. * Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm: (-[WKContentView requestPreferredArrowDirectionForEditMenuWithCompletionHandler:]): In the case where there are clickable controls above the selection, use `UIEditMenuArrowDirectionUp` to make the callout bar present _below_ the selection instead of above; otherwise, simply go with the default behavior, which puts the callout bar above the selection. (-[WKContentView requestRectsToEvadeForSelectionCommandsWithCompletionHandler:]): Pull out common logic for requesting a list of rects to evade into a separate internal helper method, and use this common helper method to implement both the legacy delegate method (which no longer works on iOS 16), as well as the new delegate method in iOS 16. For now, I opted to still implement both methods, such that this test will pass on both iOS 15 and iOS 16 (but only with the changes in rdar://95652872). (-[WKContentView _requestEvasionRectsAboveSelectionIfNeeded:]): * Tools/TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm: Also, adopt the new SPI in an API test that simulates callout bar appearance on iOS 16. (TestWebKitAPI::simulateCalloutBarAppearance): * Tools/TestWebKitAPI/cocoa/TestWKWebView.h: * Tools/TestWebKitAPI/ios/UIKitSPI.h: * Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm: (WTR::internalClassNamed): (WTR::UIScriptControllerIOS::menuRect const): (WTR::UIScriptControllerIOS::contextMenuRect const): Additionally tweak a couple of script controller hooks, to work with the new `UIEditMenuInteraction` -based callout bars on iOS 16. Canonical link: https://commits.webkit.org/251756@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295751 268f45cc-cd09-0410-ab3c-d52691b4dbfc
…k bundle directories Unreviewed build fix. Prefix a few destination paths with $(WK_FRAMEWORK_VERSION_PREFIX). The build system needs to know the real paths these items are copied to for it to be able to schedule them ahead of tasks which depend on them. * Source/WebCore/WebCore.xcodeproj/project.pbxproj: * Source/WebKit/WebKit.xcodeproj/project.pbxproj: * Source/WebKitLegacy/WebKitLegacy.xcodeproj/project.pbxproj: Remove "Copy Mig Files into Private Framework Headers" because it has no members. The files were removed in https://bugs.webkit.org/show_bug.cgi?id=232462. Canonical link: https://commits.webkit.org/251757@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295752 268f45cc-cd09-0410-ab3c-d52691b4dbfc
…crypted-pdf-as-object-and-embed.html is a flaky crash https://bugs.webkit.org/show_bug.cgi?id=241879 <rdar://95728601> Unreviewed test gardening. * LayoutTests/platform/mac-wk1/TestExpectations: Canonical link: https://commits.webkit.org/251758@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295753 268f45cc-cd09-0410-ab3c-d52691b4dbfc
…s in ShareableBitmapCG.cpp https://bugs.webkit.org/show_bug.cgi?id=241876 Unreviewed build fix. * Source/WebKit/Shared/cg/ShareableBitmapCG.cpp: (WebKit::bitmapInfo): Add some casts. Canonical link: https://commits.webkit.org/251759@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295754 268f45cc-cd09-0410-ab3c-d52691b4dbfc
https://bugs.webkit.org/show_bug.cgi?id=241713 Unreviewed build fix. Product Dependencies phases must be set to the "Products Directory" destination. Otherwise, they unintentionally embed the dependencies into their target's build product. This one must have been a default setting that got overlooked. * Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: Canonical link: https://commits.webkit.org/251760@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295755 268f45cc-cd09-0410-ab3c-d52691b4dbfc
…copyDisplayedPixelBuffer] https://bugs.webkit.org/show_bug.cgi?id=241788 rdar://94325004 Reviewed by Eric Carlson. In r292811, we enabled MSE inline painting on iOS 16 and macOS Ventura; this was intentionally limited to these versions, since CoreMedia made refinements in these OS versions to prune the `AVSampleBufferVideoOutput` queue more frequently, in order to avoid a large increase in memory use while playing MSE videos, due to accumulating excess video output frame data. However, this more frequent pruning interval has led to significantly increased power use when playing MSE video, due to the extra work done every time the pruning timer fires. To ensure that Live Text in MSE video and MSE to canvas painting still work in iOS 16 and macOS Ventura, we instead adopt new AVFoundation SPI that allows us to ask `AVSampleBufferDisplayLayer` directly for the currently displayed pixel buffer. As opposed to the `AVSampleBufferVideoOutput`- based approach, this will only kick in if MSE video inline painting is actually requested (either by the page, or from within the engine, in the case of Live Text), which avoids both increased memory use and power use. On versions of macOS and iOS that don't have the new SPI, we simply fall back to the `AVSampleBufferVideoOutput`-based snapshotting approach that we currently use. We also fall back to using the video output if the display layer is empty, in which case the backing `CAImageQueue` won't contain _any_ displayed surfaces (which means `-copyDisplayedPixelBuffer` will always end up returning null). By refactoring logic to create and set `m_videoOutput` out into a helper method (`updateVideoOutput`) that's invoked after we've finished setting up the sample buffer display layer, we can transition as needed between setting and unsetting the video output, based on whether or not the display layer is actually displaying any content. There should be no change in behavior, apart from less memory and power use due to not spinning up the `AVSampleBufferVideoOutput` queue whenever we play MSE videos. See below for more details. * Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml: Gate MSE inline painting on `HAVE(AVSAMPLEBUFFERDISPLAYLAYER_COPYDISPLAYEDPIXELBUFFER)`, instead of `HAVE(LOW_AV_SAMPLE_BUFFER_PRUNING_INTERVAL)`. * Source/WTF/wtf/PlatformHave.h: Add a new feature flag to guard the availability of the new AVFoundation SPI, `-[AVSampleBufferDisplayLayer copyDisplayedPixelBuffer]`. * Source/WebCore/PAL/pal/spi/cocoa/AVFoundationSPI.h: Add a staging declaration for `-copyDisplayedPixelBuffer`, so that we can maintain source compatibility when building against older versions of the iOS 16 or macOS Ventura SDKs. * Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp: (WebCore::WebGLRenderingContextBase::texImageSourceHelper): * Source/WebCore/platform/graphics/MediaPlayer.cpp: * Source/WebCore/platform/graphics/MediaPlayer.h: * Source/WebCore/platform/graphics/MediaPlayerPrivate.h: Replace more uses of `HAVE(LOW_AV_SAMPLE_BUFFER_PRUNING_INTERVAL)` with the new flag `HAVE(AVSAMPLEBUFFERDISPLAYLAYER_COPYDISPLAYEDPIXELBUFFER)`, which is now used to guard availability of MSE inline painting. The purpose of guarding this logic behind `!HAVE(LOW_AV_SAMPLE_BUFFER_PRUNING_INTERVAL)` in the first place seems to have been to limit the `willBeAskedToPaintGL()` codepaths to versions of macOS and iOS, where we can't enable MSE inline painting due to lack of system support. Since "system support" now depends on the availability of `-copyDisplayedPixelBuffer`, we should change to use that flag instead of one about pruning interval frequency. This also allows us to remove the `HAVE(LOW_AV_SAMPLE_BUFFER_PRUNING_INTERVAL)` flag altogether, now that there isn't any code that needs to be guarded by it. * Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h: * Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm: (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastPixelBuffer): Adjust this logic to ask `m_sampleBufferDisplayLayer` for a copy of the last displayed pixel buffer, instead of grabbing it from the video output, if possible. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::shouldEnsureLayer const): (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged): (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateVideoOutput): Factor out logic for creating or destroying the video output into a separate helper method, that's invoked after updating the display layer. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayer): (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::readbackMethod const): Replace `isVideoOutputAvailable()` with another helper method, that returns a strongly typed enum indicating which readback method to use. `None` indicates that readback isn't supported, `CopyPixelBufferFromDisplayLayer` indicates that we'll use the new AVFoundation SPI method, and `UseVideoOutput` indicates that we'll fall back to `AVSampleBufferVideoOutput`. (WebCore::MediaPlayerPrivateMediaSourceAVFObjC::isVideoOutputAvailable const): Deleted. * Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h: * Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in: * Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm: * Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp: * Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h: Canonical link: https://commits.webkit.org/251761@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295756 268f45cc-cd09-0410-ab3c-d52691b4dbfc
rdar://87794804 Reviewed by Brent Fulgham * Source/WebCore/PAL/pal/spi/cg/CoreGraphicsSPI.h * Source/WebCore/page/ActivityState.cpp * Source/WebCore/page/ActivityState.h * Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivateForTesting.h * Source/WebKit/UIProcess/API/Cocoa/WKWebViewTesting.mm * Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm * Source/WebKit/UIProcess/Cocoa/WebProcessProxyCocoa.mm * Source/WebKit/UIProcess/WebPageProxy.cpp * Source/WebKit/UIProcess/WebPageProxy.h * Source/WebKit/UIProcess/WebProcessPool.h * Source/WebKit/UIProcess/WebProcessProxy.h * Source/WebKit/UIProcess/mac/WindowServerConnection.h * Source/WebKit/UIProcess/mac/WindowServerConnection.mm * Tools/TestWebKitAPI/Tests/WebKit/GetUserMedia.mm Canonical link: https://commits.webkit.org/251762@main git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295757 268f45cc-cd09-0410-ab3c-d52691b4dbfc
webkit-early-warning-system
force-pushed
the
mute-capture-when-disconnected-from-hardware-console
branch
from
2a37553 to
6e14685
Compare
|
Committed r295757 (251762@main): https://commits.webkit.org/251762@main Reviewed commits have been landed. Closing PR #1691 and removing active labels. |
webkit-commit-queue
removed
the
merge-queue
yury-s
pushed a commit
to yury-s/WebKit
that referenced
this pull request
Jul 28, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
6e14685