Puppeteer takes partial page screenshot instead of full page

[petarthewizard]

I am trying to take a full page screenshot of http://digg.com (mobile version) but it always takes a partial screenshot, it also happens on https://thenextweb.com/. Here is the code I am using:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('http://digg.com');
    await page.setViewport({width: 320, height: 480});

    await page.waitForNavigation({'waitUntil' : 'networkidle'});

    await page.screenshot({path: 'digg.png', fullPage: true});

    await browser.close();
})();

[udaykanthr]

Hi,

Please try with,
await page.goto(url, {waitUntil: 'networkidle0', timeout: 60000});
await page.waitFor(60000); (instead of waitForNavigation())

I believe this issue is with "waitUntil: 'networkidle0'" where it is not working as expected (wait until all page request -> response completes)

[dmytroboiko]

Hi, @udaykanthr
I tried:

await page.goto(url, {waitUntil: 'networkidle0', timeout: 60000, networkIdleTimeout: 1000 * 3});
await page.waitFor(60000);
screenshot = await page.screenshot({path: 'test.png', fullPage: true});

For site, http://www.spiegel.de/ screenshot repeated some contents and doesn't have full content of page screenshot
For http://digg.com have full page, but some images weren't uploaded screenshot

[marijanlekic]

@dmytroboiko

Regarding problem with images not being loaded. Im not sure if this is connected to your problem because i do not have time to try it now but there are some sites that are not loading images unless you scroll the page. Especially the modern sites that have a lot of images on the page (the reason why is that is pretty clear), i had almost the similar situation where i made some scroll function and everything worked fine... (maybe you can combine it with network requests also to be sure that its loaded)

[petarthewizard]

@udaykanthr I am still getting the same problem, note that I am using the viewport option in my code.

[shanecp]

I see the same problem.

On code I have it as,
await page.screenshot({path: 'results.png', fullPage: true});

This what I see on the terminal when custom viewport settings are given.

puppeteer:page page.screenshot({"path":"\"results.png\""}) +5s
puppeteer:page page.setViewport({"width":"1200","height":"1000"}) +592ms

Maybe when you have a custom viewport it overrides the fullPage parameter?

[dmytroboiko]

In my code I don't use setViewport, I use default values

[petarthewizard]

@udaykanthr I tried this using "networkidle2" in the 0.13 release but I still get the same result : (

[udaykanthr]

@petarvasilev91,

looks like this issue can now be resolved with the latest fix #1173 in version 0.13
Please try using waitForNavigation() again.

NOTE: until now i tried using timeout for 60Secs, for taking screenshots for page with images and videos, which worked as expected, now i will try change my code to use waitForNavigation().

[petarthewizard]

@udaykanthr with waitForNavigation() now I get timeout exceeded. Here is the code:

`
const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('http://digg.com', {waitUntil: 'networkidle2', timeout: 60000});
await page.setViewport({width: 320, height: 480});

await page.waitForNavigation({waitUntil: 'networkidle2'});

await page.screenshot({path: 'digg.png', fullPage: true});

await browser.close();

})();
`

[fangqiao]

@marijanlekic
Hi, I am trying to convert some HTML page to PDF, which used lazy loading of images, as exactly as you said in your comment.

And you said

i had almost the similar situation where i made some scroll function and everything worked fine...

Can you share your code of scroll function?

Many thanks

[petarthewizard]

@fangqiao look here:

https://bitbucket.org/petarvasilev/puppeteer-screenshots/src/8fb1909b81c7cca96c5486b2928e2ea580c2171b/screenshot.js?at=master&fileviewer=file-view-default

starting at line 83

[fangqiao]

@petarvasilev91 Thanks. It works well, except that the latest puppeteer has a few changes of api that need to be applied to your code.

[warrell]

We have an Angular Single Page App that sets up the Angular environment and then navigates between different views. The reason that the snapshot was not taken (and the Promise was failing) was because the second page was different to the first. So the condition in the NavigationWatcher.js needed changing from this._frame._loaderId === this._initialLoaderId to !this._frame._loaderId === this._initialLoaderId

  _checkLifecycleComplete() {
    // We expect navigation to commit.
    if (!this._frame._loaderId === this._initialLoaderId)
      return;
    if (!checkLifecycle(this._frame, this._expectedLifecycle))
      return;
    this._lifecycleCompleteCallback();

    /**
     * @param {!Puppeteer.Frame} frame
     * @param {!Array<string>} expectedLifecycle
     * @return {boolean}
     */
    function checkLifecycle(frame, expectedLifecycle) {
      for (const event of expectedLifecycle) {
        if (!frame._lifecycleEvents.has(event))
          return false;
      }
      for (const child of frame.childFrames()) {
        if (!checkLifecycle(child, expectedLifecycle))
          return false;
      }
      return true;
    }
  }

[intellix]

Also experiencing fullPage: true doing nothing. Tried setting and not setting viewport but it seems not to have an effect. Thinking for now a workaround will be to scroll full heights and take screenshots until all of the page is done

Edit: It's probably cause I created a fake scrolling element using an overflowed element so the page didn't actually have height

[jessehattabaugh]

I am experiencing this issue, but only when running headfully. See I use WSL and though I was able to get puppeteer to work by specifying the path to Windows Chrome, if I tried to run it headlessly it always times out with no error. It works okay with "headless: false" so I just rolled with that because it doesn't matter to me. Then I captured a screenshot, and that worked too. But when I used setViewport() to emulate a mobile device the screenshots come out entirely transparent. I tried passing different args to setViewport, tried waitFor(10000) in various places. Tried slowmo. Nothing worked, so I decided to try it outside of WSL, but the behavior was the same. Then I tried setting "headless:true" and sure enough it works fine. So I'm guessing the root of the issue is some kind of race between setViewport and screenshot but only when running headfully.

[ratkorle]

Due to viewport jumping all the time when:
await page.screenshot({fullPage: true});
I can't manage to send clicks by x, y.

Is there I way to find the viewport dimensions when puppeteer takes screenshot ?

[br-kwon]

Anyone still experiencing this?
Try going to about:blank first after setting viewport.

  const desiredUrl = 'http://cnn.com'
  const page = await browser.newPage();
  await page.setViewport({width: 320, height: 480});
  await page.goto('about:blank');
  await page.goto(desiredUrl);

[ernestgwilsonii]

(bump) Same here, still not getting full screen caps on very long pages.

[aslushnikov]

We're hitting the GPU memory limit on the chromium side. Instead of returning corrupted screenshots, we probably need to limit screenshot height to 2^14 and notify users that very long pages won't work.

[DangerLifter]

I also had partial screenshots issue on Centos7 + Chrome v.75. Downgrade chrome version to stable v.71 solved issue in my case.

[petarthewizard]

@udaykanthr using 1.18.1 and this code:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://thenextweb.com/');
    await page.setViewport({width: 320, height: 480});

    await page.screenshot({path: 'next.png', fullPage: true});

    await browser.close();
})();

it still takes a partial screenshot for some reason...

[leodutra]

+1 for clipping
on Ubuntu 18.04 and a long table like https://coinmarketcap.com

[leodutra]

Looks like it is really memory related.
Taking 2 screenshots of every table 50 rows resolved.

[leodutra]

For testing:

const puppeteer = require('puppeteer')

async function main() {
    const browser = await puppeteer.launch({
        args: ['--start-maximized'],
        headless: false,
        defaultViewport: null
    })
    
    const page = (await browser.pages())[0]
    await page.goto('https://coinmarketcap.com/', { waitUntil: 'networkidle0' })
    await page.evaluate(() => document.querySelector('.banner-alert-fixed-bottom').remove())
    await screenshotDOMElements({
        page,
        path: 'element.png',
        selector: 'table.dataTable > tbody > tr:nth-child(-n+50)',
        addHeight: true
    })
    await screenshotDOMElements({
        page,
        path: 'element2.png',
        selector: 'table.dataTable > tbody > tr:nth-child(n+50)',
        addHeight: true
    })
    console.log('Finished')
}

main()
    .catch(error => {
        console.error(error)
        process.exit(1)
    })

/**
 * Takes a screenshot of a DOM elements on the page, with optional padding.
 * @return {!Promise<!Buffer>}
 */
async function screenshotDOMElements({
    page,
    padding = 0,
    path = null,
    selector,
    addWidth = false,
    addHeight = false
} = {}) {

    if (!selector) {
        throw Error('Please provide a selector.')
    }
    
    const rect = await page.evaluate((selector, addWidth, addHeight) => {
        let minX = Infinity
        let minY = Infinity
        let maxWidth = 0
        let maxHeight = 0
        const elements = document.querySelectorAll(selector)
        if (elements) {
            for (const element of elements) {
                const {x, y, width, height} = element.getBoundingClientRect()
                minX = Math.min(minX, x)
                minY = Math.min(minY, y)
                maxWidth = addWidth
                    ? maxWidth + width
                    : Math.max(maxWidth, width)
                maxHeight = addHeight
                    ? maxHeight + height
                    : Math.max(maxHeight, height)
            }
            return {
                left: minX,
                top: minY, 
                width: maxWidth, 
                height: maxHeight 
            }
        }
        return null
    }, selector, addWidth, addHeight)

    if (!rect) {
        throw Error(`Could not find element that matches selector: ${selector}.`)
    }
        
    return page.screenshot({
        path,
        clip: {
            x: rect.left - padding,
            y: rect.top - padding,
            width: rect.width + padding * 2,
            height: rect.height + padding * 2
        }
    })
}

[faqndo97]

I resolve the problem of the screenshot getting the html element dimensions and passing it to the clip argument on the screenshot method, and setting the x and y options at 0,0

let [height, width] = await page.evaluate(() => {
  return [
    document.getElementsByTagName('html')[0].offsetHeight,
    document.getElementsByTagName('html')[0].offsetWidth
  ]
})

await page.screenshot({
  path: './screenshots/home.png',
  clip: { x: 0, y: 0, width, height }
})

You will need consider edit the viewport for get a web or mobile view.

[leodutra]

@faqndo97 this does not solve for large elements. That's the particularity of this issue.

[aprilandjan]

Just use page.$(selector)(or page.$$(selector)) to get elementHandle and then execute snapshot method on it:

const el = await page.$('.box');
await el.snapshot({
  path: './images/box.png'
})

[AlexandrMel]

I solved same problem with enlargement of the viewport:
await page.setViewport({
width: 1200,
height: 4600
})

i needed several pages so i huge height got me my screenshots,
Goodluck

[huanzochen]

I resolve the problem of the screenshot getting the html element dimensions and passing it to the clip argument on the screenshot method, and setting the x and y options at 0,0

let [height, width] = await page.evaluate(() => {
  return [
    document.getElementsByTagName('html')[0].offsetHeight,
    document.getElementsByTagName('html')[0].offsetWidth
  ]
})

await page.screenshot({
  path: './screenshots/home.png',
  clip: { x: 0, y: 0, width, height }
})

You will need consider edit the viewport for get a web or mobile view.

Thanks for your help !
I need max element height to replace the shotSize: { height: 'all' } by webshot in npm

let contentHeight = await page.evaluate(() => {
   return document.getElementsByTagName('html')[0].offsetHeight;
})

is useful for me !

[amitesh-rai]

I found a solution which is applicable to multiple sites using the page.setViewPort(...) method as given below:

const puppeteer = require('puppeteer');

async(() => {
    const browser = await puppeteer.launch({
        headless: true, // Set to false while development
        defaultViewport: null,
        args: [
            '--no-sandbox',
            '--start-maximized', // Start in maximized state
        ],
    });

    const page = await = browser.newPage();
    await page.goto('https://www.digg.com/', {
        waitUntil: 'networkidle0', timeout: 0
    });

    // Get scroll width and height of the rendered page and set viewport
    const bodyWidth = await page.evaluate(() => document.body.scrollWidth);
    const bodyHeight = await page.evaluate(() => document.body.scrollHeight);
    await page.setViewport({ width: bodyWidth, height: bodyHeight });

    await page.waitFor(1000);
    await page.screenshot({path: 'digg-example.png' });
})();

[shivendrahandysolver]

This code is working for me

const puppeteer = require('puppeteer');

async function run() {
let browser = await puppeteer.launch();
let page = await browser.newPage();
await page.setViewport({ width: 1366, height: 766 });
await page.goto('https://modestdev.com/posts/install-composer-on-ubuntu-linux');
await page.screenshot({ path: './image.png', fullPage: true });
browser.close();
}

run();

source: https://www.scrapehero.com/how-to-take-screenshots-of-a-web-page-using-puppeteer/

[guest271314]

One issue is Chromium returns value for document.body.clientHeight that is less than the actual height of the document.

For example, at Chromium 87, navigating to MDN https://developer.mozilla.org/en-US/ and running document.body.clientHeight the output is 2818, at Firefox 81 and Nightly 83 each outputs 2857.

At DevTools CTRL+SHIFT+P running the command "Capture full size screenshot" will result in an image 39 less vertical pixels than expected.

[oczkowski]

If anyone is having this issue still I have done the following to get a full-page screenshot.

1. Launch the browser with higher window size. In this case, I matched my screen size.
   browser = await puppeteer.launch({ product: 'chrome', args: [--window-size=${windowWidth},${windowHeight}] });

2. Set the viewport
   await page.setViewport({ width: 1920, height: 1080 });

3. Take a screenshot with the fullPage property.
   await page.screenshot({ path: screenshot.png, fullPage: true, });

[guest271314]

I filed a Chrome bug for this https://bugs.chromium.org/p/chromium/issues/detail?id=1164757.

[loureirorg]

This code works for me:

const puppeteer = require('puppeteer');

(async() => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    // Set width.
    await page.setViewport({ width: 1920, height: 1 });

    // Load page.
    await page.goto('https://digg.com/', {
        waitUntil: 'networkidle0', timeout: 0
    });

    // Get scroll height of the rendered page and set viewport
    const bodyHeight = await page.evaluate(() => document.body.scrollHeight);
    await page.setViewport({ width: 1920, height: bodyHeight });

    // Screenshot and exit.
    await page.screenshot({ path: 'screenshot.png', fullPage: true });
    await browser.close();
})();

[Underknowledge]

From what I can tell, I think the problem could be solved by a setting to throttle the screenshot process.
The browser or the connection might just not be fast enough to load all the graphics needed or to render graphs.

mock-up be like

    await page.screenshot({ 
      path: 'home_assistant.png', 
      fullPage: true, 
      delay: '500ms' // just wait between the fullPage resize and the actual screenshot creation
    });

[stale]

We're marking this issue as unconfirmed because it has not had recent activity and we weren't able to confirm it yet. It will be closed if no further activity occurs within the next 30 days.

[stale]

We are closing this issue. If the issue still persists in the latest version of Puppeteer, please reopen the issue and update the description. We will try our best to accomodate it!