-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[macOS] Add support for QR code detection when showing a context menu #10709
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pxlcoder
added
the
Platform
Portability improvements and other general platform improvements not driven directly by site bugs.
label
Feb 27, 2023
EWS run on previous version of this PR (hash 177aea4) |
EWS run on previous version of this PR (hash ca0950f) |
pxlcoder
requested review from
whsieh,
megangardner,
rr-codes,
smfr and
hortont424
February 27, 2023 17:17
whsieh
approved these changes
Feb 27, 2023
Thanks for the review! |
https://bugs.webkit.org/show_bug.cgi?id=252971 rdar://105953011 Reviewed by Wenson Hsieh. Safari currently implements support for QR code detection when showing a context menu using an injected bundle. This patch works towards two goals by adding equivalent support at the WebKit layer: 1. Reducing the surface of Safari's injected bundle. 2. Enabling iokit blocking in the Web Process. Safari's QR code detection is implemented using the Vision framework in the Web Process. With iokit blocking enabled, this approach will fail, as Vision fails to create `CIImage`s. This patch adds initial support for QR code detection when showing a context menu in WebKit. Support is implemented by using the Vision framework in the UIProcess and is controlled by new SPI on `WKWebViewConfiguration`, as QR code detection is not inexpensive. Note that Safari's heuristics for detecting QR codes outside of <img> elements are currently unsupported. These heuristics will be implemented in a subsequent patch. * Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml: Add an off-by-default setting for QR code detection when showing a context menu. * Source/WTF/wtf/PlatformEnableCocoa.h: * Source/WTF/wtf/PlatformHave.h: The Vision framework is available on all Cocoa platforms other than watchOS. * Source/WebCore/PAL/PAL.xcodeproj/project.pbxproj: * Source/WebCore/PAL/pal/PlatformMac.cmake: * Source/WebCore/PAL/pal/cocoa/VisionSoftLink.h: Added. * Source/WebCore/PAL/pal/cocoa/VisionSoftLink.mm: Added. * Source/WebKit/Platform/cocoa/ImageAnalysisUtilities.h: * Source/WebKit/Platform/cocoa/ImageAnalysisUtilities.mm: (WebKit::imageFilledWithWhiteBackground): Adjust the image used to perform QR code detection to have a white background to support transparent images. (WebKit::requestPayloadForQRCode): Add a helper method to obtain QR code data from a `CGImageRef`. * Source/WebKit/Shared/ContextMenuContextData.h: (WebKit::ContextMenuContextData::qrCodePayloadString const): (WebKit::ContextMenuContextData::setQRCodePayloadString): * Source/WebKit/UIProcess/API/APIContextMenuClient.h: (API::ContextMenuClient::menuFromProposedMenu): Adjust the method to take a `ContextMenuContextData` rather than a `WebHitTestResultData` to include additional context. * Source/WebKit/UIProcess/API/APIContextMenuElementInfoMac.h: * Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm: (-[WKWebView _setupPageConfiguration:]): * Source/WebKit/UIProcess/API/Cocoa/WKWebViewConfiguration.mm: (-[WKWebViewConfiguration init]): (-[WKWebViewConfiguration copyWithZone:]): (-[WKWebViewConfiguration _contextMenuQRCodeDetectionEnabled]): (-[WKWebViewConfiguration _setContextMenuQRCodeDetectionEnabled:]): * Source/WebKit/UIProcess/API/Cocoa/WKWebViewConfigurationPrivate.h: Add SPI to control whether QR code detection is performed when showing a context menu. Detection is not performed by default. * Source/WebKit/UIProcess/API/Cocoa/_WKContextMenuElementInfo.h: Expose the detection result via SPI on `_WKContextMenuElementInfo`. * Source/WebKit/UIProcess/API/Cocoa/_WKContextMenuElementInfo.mm: (-[_WKContextMenuElementInfo qrCodePayloadString]): * Source/WebKit/UIProcess/Cocoa/UIDelegate.h: * Source/WebKit/UIProcess/Cocoa/UIDelegate.mm: (WebKit::UIDelegate::ContextMenuClient::menuFromProposedMenu): * Source/WebKit/UIProcess/WebContextMenuProxy.h: Remove the `const` qualifier from the `ContextMenuContextData` member, as it can be modified once QR code detection is completed. * Source/WebKit/UIProcess/mac/WebContextMenuProxyMac.h: * Source/WebKit/UIProcess/mac/WebContextMenuProxyMac.mm: (WebKit::WebContextMenuProxyMac::show): Display the context menu asynchronously if any post-processing (QR code detection) needs to be performed. (WebKit::WebContextMenuProxyMac::showAfterPostProcessingContextData): QR code detection is only performed if the setting is enabled, and there is no link information in the hit test result. This mirrors the existing behavior in Safari. (WebKit::WebContextMenuProxyMac::useContextMenuItems): * Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: * Tools/TestWebKitAPI/Tests/WebKit/qr-code.png: Added. * Tools/TestWebKitAPI/Tests/mac/ContextMenuTests.mm: (TestWebKitAPI::TEST): Added tests for QR code detection when showing a context menu. Canonical link: https://commits.webkit.org/260897@main
webkit-early-warning-system
force-pushed
the
eng/252971
branch
from
February 27, 2023 20:12
4867e83
to
b8d16fc
Compare
Committed 260897@main (b8d16fc): https://commits.webkit.org/260897@main Reviewed commits have been landed. Closing PR #10709 and removing active labels. |
webkit-commit-queue
removed
the
merge-queue
Applied to send a pull request to merge-queue
label
Feb 27, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Platform
Portability improvements and other general platform improvements not driven directly by site bugs.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
b8d16fc
4867e83
π wincairoπ§ͺ ios-wk2π§ͺ api-macπ§ͺ gtk-wk2π§ͺ api-iosπ§ͺ mac-wk1π§ͺ api-gtkπ π§ͺ jscπ§ͺ mac-wk2π tv-simπ§ͺ mac-AS-debug-wk2π watchπ§ͺ mac-wk2-stress