[Bug]: script tossing DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned. #11094
Comments
dwillisgh
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
github-actions
bot
added
needs-feedback
not-reproducible
and removed
invalid
not-reproducible
needs-feedback
labels
|
I have no idea why this bug post is invalid. :-) |
|
Hi. I am facing the same error and the example is reproducible in my environment. |
|
I think we need a way to make XPath queries be atomic. |
|
Still facing the issue const getProductsData = async (
page: Page,
url: string
): Promise<ProductData[]> => {
await page.goto(url)
const body = await page.$("body")
const innerText = await (await body?.getProperty("innerText"))?.jsonValue()
if (innerText?.toLowerCase().includes("oops!")) {
console.log("oops!")
await utils.sleep(3000)
return await getProductsData(page, url)
}
const waitedEl = await page.waitForXPath(selectors.amazon.xpath.resultItem)
console.log({ waitedEl: waitedEl })
const elements = await page.$x(selectors.amazon.xpath.resultItem)
console.log({ elements: elements })
const scrapedData: ProductData[] = []
return scrapedData
}
puppeteer
.use(StealthPlugin())
.launch({ headless: false })
.then(async (browser) => {
const page = await browser.newPage()
const results = await amazon.getProductsData(
page,
"https://www.amazon.in/s?i=fashion&rh=n%3A6648217031%2Cn%3A11987349031%2Cp_85%3A10440599031&dc&ascsubtag=19bBo6j6&cat=7549&d_f_save=0-100&dealsby=dealsmagnet&filter-by=0&primestatus=yes&qid=1696589137&rnid=6648217031&shortby=0&tag=worlds-most-advanced-deal-finder-21&topbrands=yes&ref=is_r_n_5&ds=v1%3Abw03pzgJ0qYfrrWdaSx4d5eRftZKkpVL7LP6u5qTXFI"
)
console.log(results)
})Getting the following error Puppeteer version: 21.3.8 |
The fix has not yet been published to npm. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Minimal, reproducible example
Error string
DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
Bug behavior
Background
We have a script that grabs urls from job sites. When I upgraded to the latest version of puppeteer
"puppeteer": "21.2.1",
I receive the following error from a few jobsites. The majority work without error.
The script has an example of a url that works and one that doesnt work.
Expectation
no error
Reality
it tossed the exception I will parrot here
/usr/local/bin/node /Users/dwillis/Documents/GitHub/jobs-node-crawler-service/issue.js
Puppeteer old Headless deprecation warning:
In the near future
headless: truewill default to the new Headless modefor Chrome instead of the old Headless implementation. For more
information, please see https://developer.chrome.com/articles/new-headless/.
Consider opting in early by passing
headless: "new"topuppeteer.launch()If you encounter any bugs, please report them to https://github.com/puppeteer/puppeteer/issues/new/choose.
page url element count: 170
node:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^
DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
U (pptr:internal:3:10576)
U.next ()
Async-from-Sync Iterator.next ()
pptr:evaluateHandle;transposeIterableHandle%20(%2FUsers%2Fdwillis%2FDocuments%2FGitHub%2Fjobs-node-crawler-service%2Fnode_modules%2Fpuppeteer-core%2Flib%2Fcjs%2Fpuppeteer%2Fcommon%2FHandleIterator.js%3A132%3A77):3:17
AsyncGenerator.next ()
pptr:evaluateHandle;fastTransposeIteratorHandle%20(%2FUsers%2Fdwillis%2FDocuments%2FGitHub%2Fjobs-node-crawler-service%2Fnode_modules%2Fpuppeteer-core%2Flib%2Fcjs%2Fpuppeteer%2Fcommon%2FHandleIterator.js%3A75%3A69):4:47
at #evaluate (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:252:55)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ExecutionContext.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:201:16)
at async IsolatedWorld.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/IsolatedWorld.js:122:16)
at async CDPJSHandle.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/api/JSHandle.js:159:20)
at async fastTransposeIteratorHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:75:54)
at async transposeIteratorHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:122:14)
at async transposeIterableHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:137:9)
at async XPathQueryHandler.queryAll (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/QueryHandler.js:122:13)
Puppeteer configuration file (if used)
No response
Puppeteer version
21.3.8
Node version
19.6.0
Package manager
npm
Package manager version
8.19.3
Operating system
macOS
The text was updated successfully, but these errors were encountered: