Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless #6210

Open
zztessema opened this issue Jan 21, 2020 · 9 comments · May be fixed by #6440
Open

Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless #6210

zztessema opened this issue Jan 21, 2020 · 9 comments · May be fixed by #6440

Comments

@zztessema
Copy link

@zztessema zztessema commented Jan 21, 2020

Current behavior:

When running cy.screenshot(), the screenshot differs when running in Chrome Headless vs Chrome Headed vs Electron Headless:

Electron Headless:
test1

Chrome Headed:
test1

Chrome Headless:
test1

Desired behavior:

It seems as though any screenshot taken while running headless chrome will produce an image that is only 800x600 pixels. This is an issue because i would like to run all tests using headless chrome, however the screenshots headless chrome produces are not large (wide/tall) enough to fully capture my application.

When running chrome headless, changing the size using cy.viewport only works when the given size is less than 800x600. For example if i call cy.viewport(1920,1080) then call cy.screenshot(), the screenshot will be 800x600. However if i call cy.viewport(700,500) then call cy.screenshot(), the screenshot will be 700x500.

Test code to reproduce

cy.visit('https://google.com/')

cy.screenshot('test1') //produces a 800x600 pixel image

cy.viewport(700,500)
cy.screenshot('test2') //produces a 700x500 pixel image

cy.viewport(1920,1080)
cy.screenshot('test3') //produces a 800x600 pixel image

Versions

Cypress 3.8.2
Windows 10
Resolution 1920x1200
node 12.13.1
npm 6.12.1

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Jan 22, 2020

I do see this sizing discrepancy noted in the original PR fixing the chrome headless here: #5953 (comment), but this is not documented anywhere in our documentation and I would definitely find this confusing as a user.

@flotwig can you comment on why there is this limitation, if there is a workaround? I'd like this documented if so.

@jennifer-shehane jennifer-shehane changed the title Cypress screenshots differ when running in Chrome Headless Cypress screenshot and video size are restricted to 800 x 600 in Chrome Headless Jan 28, 2020
@flotwig flotwig self-assigned this Jan 28, 2020
@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Jan 28, 2020

I think there was a gap in the tests for screenshots taken via Chrome headless, because I have not noticed this before.

I wonder if #5899 would fix this.

@Threnos

This comment has been minimized.

Copy link

@Threnos Threnos commented Jan 29, 2020

Faced this issue as well.
Debugging failing headless tests became harder - such small wiewport prevents from reading error messages.

@steveharrison

This comment has been minimized.

Copy link

@steveharrison steveharrison commented Jan 30, 2020

I fixed this issue by overriding the Chrome window size with the largest viewport used in my tests (in my case, the Cypress default viewport of 1000x660px) in the before:browser:launch hook.

plugins/index.js

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--window-size=1000,660');
      return args;
    }
});

I can also just run Chrome without --headless inside Docker, because then the Chrome window size will be 1050x1004px.

I am a little bit confused about how the viewport works in Cypress though: I've read stuff about the XVFB viewport, Chrome viewport, and differences between headed/headless. From a user perspective, shouldn't Cypress make sure the XVFB/Browser viewport always matches what has been set in cy.viewport? If the XVFB/Browser viewport can only be set when starting up, could Cypress scan for the largest cy.viewport value and automatically use that value?

@Threnos

This comment has been minimized.

Copy link

@Threnos Threnos commented Jan 30, 2020

@steveharrison I'd say, that there's no need for such complex actions like scanning and guessing which resolution is larger etc. The most simple and sturdy solution would be to provide configurations for browser window size, e.g. browserWidth/Height.

@cooleiwhistles

This comment has been minimized.

Copy link

@cooleiwhistles cooleiwhistles commented Jan 30, 2020

I fixed this issue by overriding the Chrome window size with the largest viewport used in my tests (in my case, the Cypress default viewport of 1000x660px) in the before:browser:launch hook.

plugins/index.js

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--window-size=1000,660');
      return args;
    }
});

I can also just run Chrome without --headless inside Docker, because then the Chrome window size will be 1050x1004px.

I am a little bit confused about how the viewport works in Cypress though: I've read stuff about the XVFB viewport, Chrome viewport, and differences between headed/headless. From a user perspective, shouldn't Cypress make sure the XVFB/Browser viewport always matches what has been set in cy.viewport? If the XVFB/Browser viewport can only be set when starting up, could Cypress scan for the largest cy.viewport value and automatically use that value?

This worked to have screenshots in expected size, but often times I see this error now (with or without the workaround)
cy.screenshot() timed out waiting '30000ms' to complete.

@0xIslamTaha

This comment has been minimized.

Copy link

@0xIslamTaha 0xIslamTaha commented Feb 3, 2020

I have this as a configuration in the plugin/index.js

  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
        args.push('--window-size=1280,1024');
  
        // whatever you return here becomes the new args
        return args
    }

But I can see that all tests gonna fail if I run it with --headless mode and here is the terminal record

asciicast

and here is the error messages in headless mode

Error: Image size (1272x720) different than saved snapshot size (1280x720).
See diff for details: /opt/code/github/cloud-fe-components/tests/visual_testing/snapshots/stepper.js/__diff_output__/test001_stepper_at_step_1.diff.png
    at Context.<anonymous> (http://localhost:6006/__cypress/tests?p=tests/visual_testing/support/index.js-495:52:17)

even trying to set the xvfb values doesn't help

xvfb-run --server-args="-screen 0, 1280x1024x24" ./node_modules/.bin/cypress run --spec tests/visual_testing/integration/stepper.js -b chrome --headless

Update

  • I run the test cases inside the docker container from cypress/browsers:node12.13.0-chrome78-ff70 images, and everything going well. here is my execution command
    npx cypress run -b chrome --headless

I'm not sure why running the --headless mode in the host machine (ubuntu 19 os), raises an error

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Feb 7, 2020

So, after talking this over with the team. This is kind of expected behavior. Basically, Chrome is running headless - which means there is no display. So, there is not a display size that Cypress can pull from and automagically know to set the window size at.

However, I advocated for having a better default than 800 x 600, which we can do. Since xvfb generally restricts the size of most users runs during cypress run to 1280 x 720, I thought this would be a better default.


@cooleiwhistles The cy.viewport() only applies to the height and width of the application under test when rendered. Not the size of the display (including the command log and Cypress Test Runner GUI).

@0xIslamTaha @cooleiwhistles I am not able to recreate any timeouts during cy.screenshot() while passing the window size using the basic Cypress screenshot command (no plugins). Are you using a plugin to take screenshots? Because this may be an issue with the plugin or it could be related to this issue #5016

Workaround

You can set the window size to whatever you want in Chrome headless.

3.x.x versions

module.exports = (on, config) => {
  on('before:browser:launch', (browser, args) => {
    if (browser.name === 'chrome' && browser.isHeadless) {
      args.push('--window-size=1280,720')
  
      return args
    }
  })
}

4.x.x version

module.exports = (on, config) => {
  on('before:browser:launch', (browser, launchOptions) => {
    if (browser.name === 'chrome' && browser.isHeadless) {
      console.log('TRUE')
      launchOptions.args.push('--window-size=1280,720')
  
      return launchOptions
    }
  })
}

Code

Default args we send to Chrome are here: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/browsers/chrome.ts#L42:L42

So, I guess you would have to somehow detect that they are running Chrome headless and push in the --window-size=1280,720 arg when that is true.

I think you would likely do this check in this method: https://github.com/cypress-io/cypress/blob/develop/packages/server/lib/browsers/chrome.ts#L341:L341

@Threnos

This comment has been minimized.

Copy link

@Threnos Threnos commented Feb 7, 2020

@jennifer-shehane Thank you so much for the workaround.
I will try it as soon as possible.

@jennifer-shehane jennifer-shehane linked a pull request that will close this issue Feb 13, 2020
0 of 5 tasks complete
@brian-mann brian-mann self-assigned this Feb 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

8 participants
You can’t perform that action at this time.