Skip to content
Permalink
Browse files
ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975

Reviewed by Tim Horton.

Teach `ImageAnalysisQueue` to recursively find all images on the page (including images in of subframe content)
and queue them for analysis. To do this, we refactor `enqueueAllImages` to call into a new recursive helper
method, `enqueueAllImagesRecursive`, to look for more candidate image elements that exist inside frame owner
elements (e.g. `iframe`).

Test: ImageAnalysisTests.AnalyzeImagesInSubframes

* page/ImageAnalysisQueue.cpp:
(WebCore::ImageAnalysisQueue::enqueueAllImages):
(WebCore::ImageAnalysisQueue::enqueueAllImagesRecursive):
* page/ImageAnalysisQueue.h:
ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975

Reviewed by Tim Horton.

Add an API test to verify that we extract and analyze images inside of subframes, in addition to images in the
main frame.

* TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html:

Canonical link: https://commits.webkit.org/250478@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@294088 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
whsieh committed May 12, 2022
1 parent 31381fa commit 8ec5f2316124b54c065809a6b7b469980e0f4876
@@ -1,3 +1,23 @@
2022-05-11 Wenson Hsieh <wenson_hsieh@apple.com>

ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975

Reviewed by Tim Horton.

Teach `ImageAnalysisQueue` to recursively find all images on the page (including images in of subframe content)
and queue them for analysis. To do this, we refactor `enqueueAllImages` to call into a new recursive helper
method, `enqueueAllImagesRecursive`, to look for more candidate image elements that exist inside frame owner
elements (e.g. `iframe`).

Test: ImageAnalysisTests.AnalyzeImagesInSubframes

* page/ImageAnalysisQueue.cpp:
(WebCore::ImageAnalysisQueue::enqueueAllImages):
(WebCore::ImageAnalysisQueue::enqueueAllImagesRecursive):
* page/ImageAnalysisQueue.h:

2022-05-11 Patrick Angle <pangle@apple.com>

Web Inspector: Parse InjectedScriptSource as a built-in to get guaranteed non-user-overriden JSC built-ins
@@ -97,8 +97,18 @@ void ImageAnalysisQueue::enqueueAllImages(Document& document, const String& iden
m_identifier = identifier;
}

enqueueAllImagesRecursive(document);
}

void ImageAnalysisQueue::enqueueAllImagesRecursive(Document& document)
{
for (auto& image : descendantsOfType<HTMLImageElement>(document))
enqueueIfNeeded(image);

for (auto& frameOwner : descendantsOfType<HTMLFrameOwnerElement>(document)) {
if (RefPtr contentDocument = frameOwner.contentDocument())
enqueueAllImagesRecursive(*contentDocument);
}
}

void ImageAnalysisQueue::resumeProcessing()
@@ -54,6 +54,8 @@ class ImageAnalysisQueue {
void resumeProcessingSoon();
void resumeProcessing();

void enqueueAllImagesRecursive(Document&);

enum class Priority : bool { Low, High };
struct Task {
WeakPtr<HTMLImageElement> element;
@@ -1,3 +1,18 @@
2022-05-11 Wenson Hsieh <wenson_hsieh@apple.com>

ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975

Reviewed by Tim Horton.

Add an API test to verify that we extract and analyze images inside of subframes, in addition to images in the
main frame.

* TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html:

2022-05-10 Ben Nham <nham@apple.com>

Re-send connection configuration if webpushd dies
@@ -255,6 +255,23 @@ static void processRequestWithResults(id, SEL, VKImageAnalyzerRequest *request,
EXPECT_WK_STREQ(overlayText, @"Foo bar");
}

TEST(ImageAnalysisTests, AnalyzeImagesInSubframes)
{
auto requestSwizzler = makeImageAnalysisRequestSwizzler(processRequestWithResults);

auto webView = createWebViewWithTextRecognitionEnhancements();
[webView synchronouslyLoadTestPageNamed:@"multiple-images"];
__block bool doneInsertingFrame = false;
[webView callAsyncJavaScript:@"appendAndLoadSubframe(source)" arguments:@{ @"source" : @"multiple-images.html" } inFrame:nil inContentWorld:WKContentWorld.pageWorld completionHandler:^(id, NSError *error) {
EXPECT_NULL(error);
doneInsertingFrame = true;
}];
Util::run(&doneInsertingFrame);

[webView _startImageAnalysis:nil];
[webView waitForImageAnalysisRequests:10];
}

TEST(ImageAnalysisTests, AnalyzeDynamicallyLoadedImages)
{
auto requestSwizzler = makeImageAnalysisRequestSwizzler(processRequestWithResults);
@@ -27,6 +27,16 @@
document.body.appendChild(image);
}

async function appendAndLoadSubframe(source)
{
const frame = document.createElement("iframe");
frame.src = source;
await new Promise(resolve => {
frame.addEventListener("load", resolve);
document.body.appendChild(frame);
});
}

function hideAllImages()
{
Array.from(document.images).forEach(image => image.style.setProperty("display", "none"));

0 comments on commit 8ec5f23

Please sign in to comment.