Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There are some missing responses on 'page.on("response") when I invoke 'page.goto()'. #2970

Closed
kbbmanse opened this issue Jul 28, 2018 · 4 comments

Comments

@kbbmanse
Copy link

  • Puppeteer version: 1.6.1
  • Platform / OS version:mac/sierra 10.12.6
  • Node.js version:8.9.0

I'm using puppeteer.connect() to reuse a browser.
I found out that some of the responses were missing when I called page.goto() again.

What steps will reproduce the problem?

const puppeteer = require('puppeteer');

const excludeResources = ['stylesheet', 'image', 'media', 'font', 'manifest', 'other'];

async function test(browser) {
    const page = await browser.newPage();
    await page.setRequestInterception(true);
    await page.setCacheEnabled(false);
    page.on('request', request => {
        if (request.resourceType() === 'document')
            console.log("request - resource type: " + request.resourceType() + ", url: " + request.url() + ", nav: " + request.isNavigationRequest() + ", redir: " + request.redirectChain().length);

        if (excludeResources.indexOf(request.resourceType()) > -1)
            request.abort('aborted');
        else
            request.continue();
    });

    page.on('response', async response => {
        if (response.request().resourceType() === 'document')
            console.log("response - resource type: " + response.request().resourceType() + ", url: " + response.url());
    });
    await page.goto('https://www.daum.net');
    await page.close();
}

puppeteer.launch({headless: false, dumpio: true}).then(async browser=> {
    console.log('First Time ---------------');
    await test(browser);
    const wsEndpoint = browser.wsEndpoint();
    browser.disconnect();

    console.log('\nSecond Time ---------------');
    puppeteer.connect({browserWSEndpoint: wsEndpoint, ignoreHTTPSErrors: true}).then(async browser=> {
        await test(browser);
        browser.close();
    });
});

execution result

First Time ---------------
request - resource type: document, url: https://www.daum.net/, nav: true, redir: 0
response - resource type: document, url: https://www.daum.net/
request - resource type: document, url: https://logins.daum.net/accounts/toploginform.do, nav: true, redir: 0
request - resource type: document, url: https://lithium.daum.net/provide/v3/shopbox, nav: true, redir: 0
request - resource type: document, url: https://shop-redapi.daum.net/red-apigate/daumtop/v3/planmall, nav: true, redir: 0
response - resource type: document, url: https://logins.daum.net/accounts/toploginform.do
response - resource type: document, url: https://lithium.daum.net/provide/v3/shopbox
response - resource type: document, url: https://shop-redapi.daum.net/red-apigate/daumtop/v3/planmall

Second Time ---------------
request - resource type: document, url: https://www.daum.net/, nav: true, redir: 0
request - resource type: document, url: https://logins.daum.net/accounts/toploginform.do, nav: true, redir: 0
response - resource type: document, url: https://logins.daum.net/accounts/toploginform.do
request - resource type: document, url: https://lithium.daum.net/provide/v3/shopbox, nav: true, redir: 0
request - resource type: document, url: https://shop-redapi.daum.net/red-apigate/daumtop/v3/planmall, nav: true, redir: 0
response - resource type: document, url: https://shop-redapi.daum.net/red-apigate/daumtop/v3/planmall
response - resource type: document, url: https://lithium.daum.net/provide/v3/shopbox

What is the expected result?
I want to get the same response every time I ask for all the requests.

What happens instead?
As you can see from the results, 'response - resource type: document, url: https://www.daum.net/' is in the first attempt, but not in the second attempt.
I think this result came out because the browser was caching the response inside.

@kbbmanse kbbmanse changed the title The responses from page.goto() do not come in normally. There are some missing responses on 'page.on("request") when I invoke 'page.goto()'. Jul 29, 2018
@kbbmanse kbbmanse changed the title There are some missing responses on 'page.on("request") when I invoke 'page.goto()'. There are some missing responses on 'page.on("response") when I invoke 'page.goto()'. Jul 29, 2018
@kbbmanse
Copy link
Author

kbbmanse commented Jul 30, 2018

The problem seems that have been solved after applying #2954 fix.

@aslushnikov
Copy link
Contributor

aslushnikov commented Jul 31, 2018

Dupe of #2970.

Edit: this is #2970.

@aslushnikov
Copy link
Contributor

Everybody: this is now fixed on tip-of-tree; it could be installed with npm i puppeteer@next.
We'll release v1.6.2 that addresses this regression shortly.

@aslushnikov aslushnikov reopened this Jul 31, 2018
aslushnikov pushed a commit to aslushnikov/puppeteer that referenced this issue Aug 1, 2018
…eer#2954)

This fixes an issue in which a response may not be recorded or trigger an event because of differing cookie headers from the request.

References puppeteer#2970
aslushnikov added a commit to aslushnikov/puppeteer that referenced this issue Aug 1, 2018
@aslushnikov
Copy link
Contributor

v1.6.2 is released to address this bug. https://pptr.dev/#?product=Puppeteer&version=v1.6.2&show=api-release-notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants