UnhandledPromiseRejectionWarning: Error: Request is already handled!

[giaphiep]

I trying use puppeteer to crawler but I have a problem when I call request to "Kick out login!". someone can help me!
thanks

`const puppeteer = require("puppeteer");
( async () => {
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.setRequestInterception(true)
await page.setUserAgent(process.env.USER_AGENT)

page.on('request', interceptedRequest => {
    interceptedRequest.continue({
    method: 'POST',
    postData: 'account=' + process.env.EMAIL + '&password=' + process.env.PASSWORD,
    headers: {
        ...interceptedRequest.headers(),
        'Content-Type': 'application/x-www-form-urlencoded',
        'Cookie': process.env.COOKIE_FAKE
        }
    })
})
const response = await page.goto(process.env.LOGIN_URL)
const body = await response.json()
const token = body.result.token
// return
if (body.result.statusCode !== 200) {
    return
} else if (body.result.type === 'login') {
    console.log('Login success!')
    // do something
}
else if (body.result.type === 'overLogin') {
    console.log('Kick out login!')
    page.on('request', interceptedRequest => {
        interceptedRequest.continue({
        method: 'POST',
        postData: 'account=' + process.env.EMAIL + '&token=' + token,
        headers: {
            ...interceptedRequest.headers(),
            'Cookie': process.env.COOKIE_FAKE
            }
        })
    })

    const response = await page.goto(process.env.LOGOUT_URL)
    let body = await response.json()

    if (body.result.statusCode !== 200) {
        return
    } else if (body.result.type === 'login') {
        console.log('Login success!')
        // do something
    } else {
        return
    }
}

})()`

[darkguy2008]

Same here, still waiting for a fix...

[mesoar]

Same here, still waiting for a fix......

[luka-dev]

I have the same issue

[davidoskay]

Same

[delmotte]

Same, any update on this ?

[mclimb]

I have the same issue as well. I uninstalled all of my puppeteer-related packages and then reinstalled each and then it worked again for a short time. Then it stopped working again... ugh

[NealBozeman]

The package puppeteer-extra-plugin-block-resources is also failing with the same error. I can't find any way to block resources using the API directly.

[rafazafar]

Hmm.. just started having this issue today. without changing any code.

[weibingzhu]

Same

[adidoes]

Happens to me too, anyone has any idea why?

[Dynom]

An answer was given here: #3853

[hooliy]

const page = await browser.newPage(); then close()
every time!

[NawarA]

Hi, this is still an open issue.

Puppeteer throws due to these lines:

HTTPRequest.js
line 217 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');
line 268 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');
line 314 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');

Disabling these hot path assertions makes everything work perfectly. Why are these assertions even in the hot path? They seem like debugging code, or perhaps test code, that was left in production. Again, removing these assertions fixes everything ... so if they're not needed for everything to work, then please drop them from the src, as they're causing issues and are observably not required.

[NawarA]

@mathiasbynens perhaps you'd know why these assertions are here? Are they a performance boost? Can they be removed? Can we have devs add a flag to skip the assertions, resolving the issue?

[AliRafieePour]

@mathiasbynens perhaps you'd know why these assertions are here? Are they a performance boost? Can they be removed? Can we have devs add a flag to skip the assertions, resolving the issue?

Seems like this approach only resolves the printing of logs not the issue itself

[NawarA]

@AliRafieePour in my dev env, commenting out the assertions allows the software to work as normal.

I recommend removing them, which solves the problem of:

• The browser instance is crashing due to Request is already handled!

As far as I can tell, "Request already handled" should be be a debug warning for developers...not something that crashes a production instance. From that point of view...yeah...commenting out those lines is good enough to solve the issue ;)

You're right, that we can say there are two issues here:

1. Dont make the browser crash when a request is handled more than once
2. Prevent browsers from handling the same request more than once

Personally, I only care about item 1, b/c it makes the software unstable. 2 can be solved at a later time, if it is an issue that causes stability issues outside of item 1.

Triage

[kirbdee]

I ended up doing something like this

page.on("request", async (req) => {
    try {
      switch (await req.resourceType()) {
        case "image":
        case "stylesheet":
        case "font":
          await req.abort();
          break;
        default:
          await req.continue();
          break;
      }
    } catch (e) {
      console.log(e);
    }
  });

you could probably get fancier, and create a store and check to make sure you're not trying to handle the same request.

but for me ultimately i didn't care just wanted to "try" to abort some files if it could

What's weird, and this is where it gets weird is, wrapping this in an async like this allows us to catch the underlying async throws even though these functions themselves are not promises/ async. not sure if that's the intended way to blanket catch under the hood async/promise errors but this seems to work...

Edit: seems like something similar was already explained #3853

[NinoSkopac]

#3853 (comment) nails it

[ptommasi]

I'm using puppeteer extra for ad and resource blocking, and I needed the "request" event for minor things (basically avoid the full page to load just to quickly push some existing state, like this).

In my case, calling page.removeAllListeners("request"); is a good workaround. It's just a patch, but request event seems to be at a page scope (browser.listenerCount("request") always return 0), and - as far as I noticed - when a new page is created, the ad and resource blockers are working again.

I noticed that I could do evil stuff, like collecting the old callback with (page as any).emitter.all.get("request")[0] (I could even check if it exists using page.listenerCount("request") > 0) to set it back later on, but then problem of context binding arises and it's just too much hassle.

It's just much simpler to create a page, remove the existing listener, do my stuff, close the page and start over. Maybe this workaround could help someone with a use case similar to mine.

[LeuKongsun]

Hi, this is still an open issue.

Puppeteer throws due to these lines:

HTTPRequest.js
line 217 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');
line 268 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');
line 314 -> assert_js_1.assert(!this._interceptionHandled, 'Request is already handled!');

Disabling these hot path assertions makes everything work perfectly. Why are these assertions even in the hot path? They seem like debugging code, or perhaps test code, that was left in production. Again, removing these assertions fixes everything ... so if they're not needed for everything to work, then please drop them from the src, as they're causing issues and are observably not required.

It works, but the problem occurs again when I deploy to firebase function. Is there any suggestion?

[otanim]

For those who are facing this issue, it's happening when you assign a listener to the same event more than twice, so you want to do is either unsubscribe your listener once you're done with the listener's code execution and only then assign a new listener:

function logRequest(interceptedRequest) {
  console.log('A request was made:', interceptedRequest.url());
}
page.on('request', logRequest);
// Sometime later...
page.off('request', logRequest);

Snipped was been copied from the official docs.

Otherwise, you always can create a new page and never unsubscribe the listener, it's up to you and to your use case, but the first option is preferable in the aspect of CPU/memory load minimization.

[hatemjaber]

Not sure why this works, but here is the solution that worked for me:

page.removeAllListeners("request");
await page.setRequestInterception(true);
page.on('request', async req => {
    ['stylesheet', 'font'].includes(req.resourceType()) ? await req.abort() : await req.continue();
});

[AroldoGoulart]

Not sure why this works, but here is the solution that worked for me:

page.removeAllListeners("request");
await page.setRequestInterception(true);
page.on('request', async req => {
    ['stylesheet', 'font'].includes(req.resourceType()) ? await req.abort() : await req.continue();
});

Not sure why too but it still working with puppeteer and puppeteer-extra with any plugins, TY! 🚀

[AroldoGoulart]

Following the approach of @hatemjaber I create a function where realize this 'middleware' to me.
Hope this helps someone.

const brewery = async (page) => {
    page.removeAllListeners("request");
    await page.setRequestInterception(true);

   // list here your block list
    const blocks_list = [
        `stylesheet`,
        `image`,
        `script`,
        `font`
    ]

    page.on('request', async req => {
        const type = req.resourceType()
        if (blocks_list.includes(type)) {
            req.abort();
        }
        else {
            await req.continue();
        }
    });
}

Just call this it with await brewery(your_page_variable) and everything shoulds works

[DustinJSilk]

Puppeteer supports multiple request handlers.

See here for more details: https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#multiple-intercept-handlers-and-asynchronous-resolutions

The tricky part was getting it working with puppeteer-extra. There is a long standing PR that adds support for it. berstend/puppeteer-extra#592

You can install that PR yourself, or use my fork where i built adblocker and puppeteer-extra plugins, if you'd like:

yarn add 'puppeteer-extra-plugin-adblocker@https://gitpkg.now.sh/DustinJSilk/puppeteer-extra/packages/puppeteer-extra-plugin-adblocker?god-damn-gitpkg-caching-3'

yarn add 'puppeteer-extra@https://gitpkg.now.sh/DustinJSilk/puppeteer-extra/packages/puppeteer-extra?god-damn-gitpkg-caching-3'

Then it becomes quite simple to solve:

page.on('request', async req => {
    const allowed = ...

    if (allowed) {
      // Only continue if it is not already handled (and thus blocked by adblocker)
      if (req.isInterceptResolutionHandled()) {
        await req.continue()
      }
    } else {
      await req.abort()
    }
  })

Its been a painful day getting this working.

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[ecmascriptguru]

The same error here as well.

error - unhandledRejection: Error: Request is already handled!
    at Object.exports.assert (/path/to/project/link-preview/node_modules/puppeteer/lib/cjs/puppeteer/common/assert.js:26:15)
    at HTTPRequest.continue (/path/to/project/link-preview/node_modules/puppeteer/lib/cjs/puppeteer/common/HTTPRequest.js:217:21)
    at PuppeteerBlocker.onRequest (/path/to/project/link-preview/node_modules/@cliqz/adblocker-puppeteer/dist/cjs/adblocker.js:262:33)
    at BlockingContext.onRequest (/path/to/project/link-preview/node_modules/@cliqz/adblocker-puppeteer/dist/cjs/adblocker.js:69:47)
    at /path/to/project/link-preview/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
    at Array.map (<anonymous>)
    at Object.emit (/path/to/project/link-preview/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
    at Page.emit (/path/to/project/link-preview/node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
    at /path/to/project/link-preview/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:143:100
    at /path/to/project/link-preview/node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62

Any further update on this issue please?

[nxvinh222]

Same error

[hatemjaber]

@ecmascriptguru @nxvinh222, I forgot how I resolved this error exactly but I think it was something like this:

page.on('request', req => {
            // stylesheet, image, media, font, script, texttrack, xhr, fetch, eventsource, websocket, manifest, other.
            if (['document'].includes(req.resourceType())) {
                return req.continue();
            }
            return req.abort();
        });

You can add other types to the array other than document if you want to check for them. I hope this helps resolve your error.

[OrKoN]

I believe this was fixed by the cooperative interception mode: see https://pptr.dev/guides/request-interception In the legacy mode, a request abort/continue can only happen once and if it happens more than once the error is thrown. If that is the case, pls check your code and see if you (or puppeteer extension modules) register multiple event handlers. If you believe there is still a bug, please file a new report with a repro.

[mrnavsingh]

Found the solution here
https://github.com/puppeteer/puppeteer/blob/59578d9cd5709bebe3117f8e060ad7cab220b3df/docs/api.md#pagesetrequestinterceptionvalue

Add this line: if (interceptedRequest.isInterceptResolutionHandled()) return;

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on('request', (interceptedRequest) => {
if (interceptedRequest.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort();
else interceptedRequest.continue();
});
await page.goto('https://example.com');
await browser.close();
})();

[mrnavsingh]

Found the solution here
https://github.com/puppeteer/puppeteer/blob/59578d9cd5709bebe3117f8e060ad7cab220b3df/docs/api.md#pagesetrequestinterceptionvalue

Add this line: if (interceptedRequest.isInterceptResolutionHandled()) return;

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on('request', (interceptedRequest) => {
if (interceptedRequest.isInterceptResolutionHandled()) return;
if (
interceptedRequest.url().endsWith('.png') ||
interceptedRequest.url().endsWith('.jpg')
)
interceptedRequest.abort();
else interceptedRequest.continue();
});
await page.goto('https://example.com');
await browser.close();
})();