New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete screenshots for large pages (6000x6000px or more) #477

Open
mstipetic opened this Issue Aug 22, 2017 · 27 comments

Comments

Projects
None yet
@mstipetic

mstipetic commented Aug 22, 2017

when I try to take a screenshot of a simple website, but in a large resolution the bottom part of the website doesn't look like it's being drawn.
You can see an example of a 6000x6000px website here: output

I've tried loading contents from disk, from a remote server, using the setContent function, going slowMo, waiting on events, focusing on different elements, disable-gpu, headless true and false (one time it succeeded properly with headless:false) etc, still with the same result (which gets worse the higher the resolution_

        const browser = await puppeteer.launch({slowMo: 500, headless: false, args: ['--disable-gpu', '--cast-initial-screen-width=6000', '--cast-initial-screen-height=6000']});
	const page = await browser.newPage();
	await page.setViewport({width: 6000, height: 6000})
	//await page.goto('file:///index_test.html', {waitUntil: 'networkidle'});
	await page.goto('http:///localhost:8000/index_test.html', {waitUntil: 'load'});
  //await page.setContent(template)
	//await buffer = page.screenshot({path: 'example.jpg', type: 'jpeg', quality: 100, fullPage: true});
	const buffer = await page.screenshot({type: 'png'});
	console.log('length:', buffer.length);
	await fs.writeFile('output.png', buffer);

	browser.close();
@Garbee

This comment has been minimized.

Contributor

Garbee commented Aug 22, 2017

Duplicate of #359

@mstipetic

This comment has been minimized.

mstipetic commented Aug 22, 2017

@Garbee are you sure it's a duplicate? I've looked at that issue before and it seems the problem there is that chrome has a hard limit on 16384px, I'm having problems on much lower resolutions than that

@samwillis

This comment has been minimized.

samwillis commented Aug 22, 2017

I have found this as well, you may be able to get round it by stitching multiple screenshots taken with the 'clip' options. It's what I was going to try before coming up against a speed issue.

@vsemozhetbyt

This comment has been minimized.

Contributor

vsemozhetbyt commented Aug 22, 2017

I recall having a similar problem with Electron. Could it be that the painting is not complete at the time of getting a screenshot? Does a timeout help?

@mstipetic

This comment has been minimized.

mstipetic commented Aug 23, 2017

@vsemozhetbyt maybe it's because the page is so large it captures it between drawing several frames, I don't know. I've tried timeouts and didn't help. My other problem also is that once using headless: false option it did work properly (but only once), I'm not sure what to make of that

@samwillis

This comment has been minimized.

samwillis commented Aug 23, 2017

Are you on a Mac and if so have you tried it on Linux?

@mstipetic

This comment has been minimized.

mstipetic commented Aug 23, 2017

@samwillis I'm on a mac. Does it perform better on linux, because I'd be running it in that environment anyways?

@aslushnikov

This comment has been minimized.

Contributor

aslushnikov commented Aug 24, 2017

await fs.writeFile('output.png', buffer);

This doesn't actually wait for the file to be written. you should use fs.writeFileSync.

@mstipetic

This comment has been minimized.

mstipetic commented Aug 28, 2017

@aslushnikov this doesn't resolve the situation, I've just tried it. Besides, the same behaviour occurs when using the 'path' argument in the screenshot function unfortunately

@aslushnikov aslushnikov reopened this Aug 28, 2017

@aslushnikov

This comment has been minimized.

Contributor

aslushnikov commented Aug 28, 2017

@mstipetic can you provide the whole script you run? I cannot reproduce this issue otherwise.

@samwillis

This comment has been minimized.

samwillis commented Aug 30, 2017

@aslushnikov I can replicate the problem this with this code:

const puppeteer = require('puppeteer');
(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.setViewport({
        width: 7000,
        height: 7000
    })
    await page.goto('about:blank');
    await page.setContent('<div style="position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: linear-gradient(to bottom, #b8e1fc 0%,#a9d2f3 10%,#90bae4 25%,#90bcea 37%,#90bff0 50%,#6ba8e5 51%,#a2daf5 83%,#bdf3fd 100%);"></div>')
    const start = Date.now();
    await page.screenshot({path: 'example.png'});
    console.log((Date.now()-start)+'ms')
    await browser.close();
})();

Interestingly if you remove the gradient from the div background, and give it a flat colour the image saves correctly with no missing areas.

I have tried it on both OSX and Ubuntu, both have the problem, saved image on osx:
example-osx

and ubuntu:
example-ubuntu

One interesting thing to note is the png seems to be being written in different size chunks on osx and ubuntu, although from a cursory glance it looks like it may be the same number of pixels that have been written...

@samwillis

This comment has been minimized.

samwillis commented Aug 30, 2017

Right, I have been playing a little more with this. The code below works (red image with about half of it covered with text) however if you remove transform: translate3d(0,0,0); or increase the font size to 150px then it stops working.

Knowing very little about how chrome works, is it composited from multiple textures? could the translate3d(0,0,0) be forcing it to composite in a certain way? Does chrome have some sort of maximum texture size internally?

const puppeteer = require('puppeteer');
(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.setViewport({
        width: 7000,
        height: 7000
    })
    await page.goto('about:blank');
    await page.setContent(`
        <body>
            <div style="transform: translate3d(0,0,0); position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: #f00; font-size: 140px;">
                Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque ultricies ut tortor vitae fermentum. Etiam vulputate diam et semper condimentum. Morbi lacinia, metus at pellentesque volutpat, sapien diam lacinia enim, non accumsan leo purus vel purus. Vestibulum aliquam eget justo sit amet sollicitudin. Morbi hendrerit id ex ac fringilla. Nulla convallis ipsum non sapien facilisis, id venenatis lectus consequat. Maecenas lacinia convallis accumsan. Aliquam id congue tortor. Vestibulum porta iaculis massa, eget congue nibh pulvinar et. Nullam in faucibus quam, eu elementum nulla.
                Nulla odio mauris, semper eu ante ac, dictum consectetur nunc. Nulla condimentum metus est, ut feugiat eros accumsan a. Aenean sagittis risus in ipsum rutrum, in tristique augue consequat. Cras sagittis ante non est consectetur tempor. In eu pretium diam, nec aliquet nisi. In ut pulvinar elit, in fermentum justo. Praesent fermentum ante in sem suscipit, et lobortis sapien porta. Ut interdum egestas arcu, sed facilisis mi. Donec elementum nibh non nunc consequat, sed tempor mi finibus. Duis pharetra nec tortor id porttitor.
                Phasellus egestas scelerisque nunc ac eleifend. Curabitur tempus rhoncus sem, id ornare nisl pellentesque auctor. Fusce ac laoreet est. Vivamus in consectetur enim. Vivamus pharetra dignissim nibh, sed porta eros imperdiet in. Fusce cursus ipsum vitae vehicula suscipit. Nulla condimentum, felis quis maximus euismod, sem elit tristique mi, non sollicitudin sapien tortor sed elit. Aliquam non iaculis quam. Suspendisse velit massa, tempus eu fermentum id, elementum volutpat metus. Nulla mattis odio eu massa convallis sodales eu eu erat. Mauris non diam ornare urna elementum molestie id vitae nibh. Maecenas purus elit, interdum eget purus sed, viverra pharetra nunc. Curabitur non elit fermentum quam ultrices dignissim. Curabitur eu nisl id enim auctor varius non eget odio.
                Nullam venenatis pellentesque nulla, ut maximus odio lobortis non. Integer a blandit justo. Etiam quis eros ipsum. Nam et risus nec nisl consequat rutrum nec eget diam. Curabitur a ipsum efficitur felis commodo sollicitudin. Aenean efficitur finibus nisl id pellentesque. Lorem ipsum dolor sit amet, consectetur adipiscing elit.
                Maecenas pretium nec metus ac sollicitudin. Suspendisse potenti. Vestibulum imperdiet massa et quam rutrum pulvinar. Pellentesque fermentum fringilla justo, quis rutrum ligula. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Mauris eleifend tincidunt placerat. Integer tincidunt tempus congue. Sed elementum eleifend velit id posuere. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Cras turpis justo, consequat ut nunc vel, rutrum convallis ex. Duis quis bibendum lacus. Sed vitae euismod lorem. Aenean fermentum elit non leo mattis, id varius mi porta. Sed semper suscipit condimentum. Vivamus in efficitur lacus, eget rhoncus dui.
            </div>
        </body>
    `)
    const start = Date.now();
    await page.screenshot({path: 'example.jpg'});
    console.log((Date.now()-start)+'ms')
    await browser.close();
})();
@samwillis

This comment has been minimized.

samwillis commented Aug 30, 2017

I have filed a bug for this on the Chromium tracker here: 760596

@jeffjacobsen

This comment has been minimized.

jeffjacobsen commented Oct 21, 2017

Any news on this? I'm having a similar problem when trying to convert svg to large png images.

I've had success generating 6000x6000 px images, but it fails at 9375x7875. The svg displays fine in Chrome browser at that size, and Phantomjs can process it. I'm running on Ubuntu. Increasing RAM doesn't help. It did get a little bit further going from 2GB to 4GB. 8GB was the same as 4GB.

@OmarAbdElNasser

This comment has been minimized.

OmarAbdElNasser commented Nov 30, 2017

Is there any solution for that problem, Please ?

@krzkaczor

This comment has been minimized.

krzkaczor commented Nov 30, 2017

@OmarAbdElNasser @jeffjacobsen I have been working on a solution for these problems: https://github.com/VisualCop/fullscreenshooter This makes multiple screenshots and stitches them together, makes sure to crop them first etc. It works with puppeter. For now, works only for long vertical pages but can be adjusted to support horizontal as well.

@OmarAbdElNasser

This comment has been minimized.

OmarAbdElNasser commented Nov 30, 2017

@krzkaczor but there will be no image for the whitespace ?

@OmarAbdElNasser

This comment has been minimized.

OmarAbdElNasser commented Nov 30, 2017

This is my code

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('D:\Image_2.HTML', {waitUntil: 'load'});
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
console.log(page.frames())
await page.screenshot({path: 'example.png',fullPage : true});

await browser.close();
})();

I made so many tests to find out why this happen.
In my case, the load event occur before finishing rendering the whole image, so if we can change waitUntil: 'load' to something like waitUntil: 'FinishRendering', it will solve the problem and i'm sure of this , the problem is that there's no option called FinishRendering, so if there's something detect the FinishingRendering instead of load, it will be great.

@krzkaczor

This comment has been minimized.

krzkaczor commented Nov 30, 2017

@OmarAbdElNasser your method (using fullPage) only works reliably for small websites. If it's bigger then there are some weird artifacts etc. That's why I have created that library.
For now, it doesn't support proper checks for "did page really loaded" but you can put timeout as you did in your snippet.

@OmarAbdElNasser

This comment has been minimized.

OmarAbdElNasser commented Nov 30, 2017

@zth

This comment has been minimized.

zth commented Dec 17, 2017

I'm also experiencing the same issue. I'm trying to export rather large (16k x 11k) PNG:s from an SVG, and going above like 5k in width/height starts bugging, with a majority of the content being white and not rendered.

I don't know if it helps, but it works fine using PhantomJS instead of Puppeteer (although the generated screenshot is huge, almost 800mb, which makes it unusable for me).

Can I do anything to help debugging this issue? Thanks!

@OmarAbdElNasser

This comment has been minimized.

OmarAbdElNasser commented Dec 17, 2017

The issue is in writing the image, so please if you can do something please do it.

@ivannluz

This comment has been minimized.

ivannluz commented Jan 23, 2018

I have a question of puppeteer how can set the browser in order to display a full page , because if a set the size of the screen does not matter how big is the screen always display just a part of the page like this
image

I don't want the big blank rectangle.

@ashirai

This comment has been minimized.

ashirai commented Jan 25, 2018

@ivannluz, I have exactly the same problem. I can set viewport size in px but not the browser size.
Ideally I just want to set browser size in % (like width:100%, height: 100%) then the viewport changes the size depending on the specified browser size. It seems to be such a basic feature, but I can't seem to find a way to do it.

@mityok

This comment has been minimized.

mityok commented Apr 30, 2018

i'm having the same tearing issue on much smaller scale (1920x1080).
was able to recreate the issue by using @samwillis examples and increasing the number of layers:
example

const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.setViewport({ width: 1920, height: 1080 }) await page.goto('about:blank'); await page.setContent('<body> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: linear-gradient(to bottom, #b8e1fc 0%,#bdf3fd 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 1%; right: 1%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 2%; right: 2%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 3%; right: 3%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 4%; right: 4%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 5%; right: 5%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 6%; right: 6%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 7%; right: 7%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 8%; right: 8%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 10%; right: 10%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 15%; right: 15%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 20%; right: 20%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 25%; right: 25%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 30%; right: 30%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 35%; right: 35%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 40%; right: 40%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 45%; right: 45%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 0; right: 0; bottom: 0; opacity:0.8;background: rgba(255,255,255,0.8)"></div> </body>') const start = Date.now(); await page.screenshot({path: 'example.png'}); console.log((Date.now()-start)+'ms') await browser.close(); })();

@zth

This comment has been minimized.

zth commented Apr 30, 2018

For anyone who's interested: My current workaround for this issue is to screenshot tiles of the viewport (about 800x800, as many as I need to cover the entire viewport) to individual PNGs, and then stitch them together using Imagemagick.

It's definitively not as easy as just taking a single large screenshot, but it works fine for my purposes. Tiles that small has worked fine so far, and I haven't had any rendering issues with this method.

@mityok

This comment has been minimized.

mityok commented Apr 30, 2018

@zth that's the direction that could work for some, unfortunately I'm taking screenshot of page with dynamic content and could have hundreds of elements, and the issue easily reproduced on small resolution of 800X600 with only 100 layers:
example
const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.setViewport({width: 800, height: 600}) await page.goto('about:blank'); await page.setContent(“ <body> <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 0; right: 0; bottom: 0; background: linear-gradient(to bottom, #b8e1fc 0%,#bdf3fd 100%);opacity:0.2;"></div> ${Array.from(Array(100)).reduce((acc,n,i)=>“${acc}<div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: ${i/2}%; right: ${i/2}%; bottom: 0; background: linear-gradient(to bottom, #f00 0%,#00f 100%);opacity:0.2;"></div>“,'')} <div style="transform: translate3d(0,0,0);position: absolute; top: 0; left: 0; right: 0; bottom: 0; opacity:0.8;background: rgba(255,255,255,0.8)"></div> </body> “) const start = Date.now(); await page.screenshot({path: 'example.png'}); console.log((Date.now()-start)+'ms') await browser.close(); })();

So i'm in need of a more robust solution, perhaps there is some arguments in chrome that could address the issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment