Skip to content

feat(InfiniteScroll): use puppeteer emulating scrolls instead of window.scrollBy#1170

Merged
B4nan merged 3 commits intomasterfrom
feat/use-puppeteer-scroll-instead-of-window-scrollBy
Sep 13, 2021
Merged

feat(InfiniteScroll): use puppeteer emulating scrolls instead of window.scrollBy#1170
B4nan merged 3 commits intomasterfrom
feat/use-puppeteer-scroll-instead-of-window-scrollBy

Conversation

@vladfrangu
Copy link
Copy Markdown
Member

This should close #1010 (but I don't know if this is the API you referred to in your issue, and it's the only other relevant example I found for scrolling). There's still a bug I need to find out why it breaks (for some reason the test case for waitForSecs: Infinity, scrollCancelCallback throws..)

@vladfrangu vladfrangu requested review from B4nan and mnmkng September 9, 2021 10:49
This gets us closer to the end-goal, but it's still far off
Copy link
Copy Markdown
Member

@B4nan B4nan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good, let's wait for approval from @mnmkng too on this one to be sure

@B4nan
Copy link
Copy Markdown
Member

B4nan commented Sep 9, 2021

Btw I came across this issue puppeteer/puppeteer#4690, that says there might be some issues with headless mode with this approach, would be good to ensure it works fine with headless (haven't checked, maybe the tests are already using headless).

(its quite old, so probably not relevant, but still...)

@vladfrangu
Copy link
Copy Markdown
Member Author

I used this script to test these changes (since jest was waay too slow for my likings), and it worked, tho I needed to increate the timeout between scrolls to make stopScrollCallback work as before

const { launchPuppeteer, utils } = require('.');

(async () => {
    const browser = await launchPuppeteer({ launchOptions: { headless: true } });

    const page = await browser.newPage();

    let count = 0;
    const content = Array(1000).fill(null).map(() => {
        return `<div style="border: 1px solid black">Div number: ${count++}</div>`;
    });
    const contentHTML = `<html><body>${content}</body></html>`;
    await page.setContent(contentHTML);

    function isAtBottom() {
        return (window.innerHeight + window.scrollY) >= document.body.offsetHeight;
    }

    const before = await page.evaluate(isAtBottom);
    console.log(before);

    await utils.puppeteer.infiniteScroll(page, { waitForSecs: Infinity, stopScrollCallback: () => true });
    // await utils.puppeteer.infiniteScroll(page, { waitForSecs: 0 });

    const after = await page.evaluate(isAtBottom);
    console.log(after);

    console.log(await page.evaluate(() => ({
        innerHeight: window.innerHeight,
        scrollY: window.scrollY,
        innerHeightPlusYOffset: window.innerHeight + window.scrollY,
        offsetHeight: document.body.offsetHeight,
    })));

    await page.close();
    await browser.close();
})();

@vladfrangu vladfrangu marked this pull request as ready for review September 10, 2021 10:10
@B4nan B4nan merged commit a22b8aa into master Sep 13, 2021
@B4nan B4nan deleted the feat/use-puppeteer-scroll-instead-of-window-scrollBy branch September 13, 2021 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve utils.puppeteer.infiniteScroll

2 participants