fix: prevent iframe expansion failure on pages with Trusted Types CSP#3590
Conversation
github-actions
Bot
added
t-tooling
barjin
left a comment
barjin
left a comment
There was a problem hiding this comment.
Thank you @nikitachapovskii-dev ! The changes seem alright to me.
I have one idea regarding better logging in suspicious cases... but it's not imo strictly necessary ⬇️
| const iframe = await frame.contentFrame(); | ||
|
|
||
| if (iframe) { | ||
| if (iframe && cheerioIframes[index]) { |
There was a problem hiding this comment.
Regarding the Playwright -> Cheerio mapping, I'm thinking we could also somehow use the srcdoc / src attributes.
Perhaps we could just compare the src(doc) from Playwright and from Cheerio in each step and log a warning if these differ? Wdyt?
There was a problem hiding this comment.
Alternatively, we could just log a warning if the iframe arrays are of different lengths
There was a problem hiding this comment.
Thanks for the suggestions!
The second option is probably better as it catches the realistic failure mode (page dynamically adds/removes an iframe) with zero overhead.
The src/srcdoc is more complex and requires an extra getAttribute call per iframe on every crawled page, while the diagnostic would almost never fire 😄
nikitachapovskii-dev
deleted the
fix/parse-with-cheerio-trusted-types-csp
branch
3 participants
Pages enforcing a Trusted Types Content Security Policy (e.g. Google Sheets) block any browser-side HTML string assignment — including innerHTML and DOMParser.parseFromString. The iframe expansion in parseWithCheerio used frame.evaluate() to inject iframe content into the browser DOM, which was silently blocked by CSP, causing iframe content to be dropped without any visible error.
The fix moves HTML assembly out of the browser entirely. page.content() is called first, loaded into Cheerio on the Node.js side, and iframe content (fetched via Playwright's Node.js API, which is unaffected by CSP) is substituted directly in the Cheerio tree.
Closes #3588