Screenshots taken during 'cypress run' do not respect the viewport size set

[egucciar]

Current behavior:

Observe these two images:

Headless / Electron:

Headed / Chrome:

Notice the spacing to the Left of our Logo is much different than the spacing to left of the Logo in headed and headless, indicating that our 1500px viewport setting was not respected in headless.

Desired behavior:

Ideally, the Viewport width should be the same for both browsers, headless and headed.

Steps to reproduce:

Try using a Website with a viewport of 1500 pixels wide and confirm if the viewport taken in headless vs headed is the same

Versions

Cypress 3.0.2 and MacOS

Motivation

I am working towards pegging down the Visual Diffing workflow for our company. Truth is, having comparable screenshots in headless vs headed runs is a desirable feature. When it comes to scaling and cropping, there is no issue on my side to do some POC work to scale/crop images and set thresholds such that differences in the CI / Headless enviroment can be accounted for and ignored. However I cannot ignore the fact that headless screenshots do not honor the viewport width I explicitly set in the Cypress.json. Ideally, if the viewport is 1500px wide headed, the headless screenshots should reflect the same.

[kuceb]

@egucciar have you tested multiple settings of the viewport width and confirmed that it is always ignored in headless electron?
Possibly related to #1857

[egucciar]

@bkucera I agree that this needs more information. I tried to output the "innerWidth" of window in both scenarios and Headless also said 1500px. Maybe it's just a difference between how High DPI screens render UIs verses the headlesS? maybe this is not a bug then, it just is confusing to me why there would be such different margins in one vs the other.

[brian-mann]

I believe the reason this is happening is because in headless mode we set XVFB to render at 1280x720.

Normally Cypress scales the screen so that running tests makes no difference to the screen size. However when taking screenshots, we no longer scale the image, which forces it to display in full width + height.

Think of the 1280x720 as the literal screen size. If you've set your viewportWidth to be 1500px then it physically cannot fit inside of 1280. Your application is responsive so that it all "works" without a scrollbar, but its not the same image anymore.

We'd need to support setting xvfb viewport via the command-line, or we'd perhaps need to issue a warning or even error if you're trying to take a cy.screenshot() at a viewport that the screen cannot actually display.

[egucciar]

Wow! Awesome that's solved for me, there's a Max width so that's good. The 1500 wasn't super necessary I'll set to 1280.

It would be nice if full length screens could be taken by sizing up the viewport rather than scrolling due to fixed header content. I wonder if there has to be some kind of boundary on the width / height in general of the view box for taking shots or why it can't be any size?

[egucciar]

@brian-mann Seeing unreliable/undesirable results with the fullPage screenshot capture. It would be awesome if the xvfb could respect the same viewport set in the Cypress.json, or an option to force it to do so, so I could use viewport as the capture setting while also ensuring the viewport is indeed large enough to capture enough of the application I care about.

[ykravv]

Hey,

I was able to make screenshots in headless electron with setting resolution as:

// plugins/index.js 
module.exports = on => {
  on('before:browser:launch', browser => {
    if (browser.name === 'electron') {
      return {
        width: 1600,
        height: 900,
      };
    }
  });
};

But the question is how to get the same effect for Chrome? Trying to get it working on CI in headless mode.

So, my config has 1600x900 viewport size.
On CI I'm getting failing screenshots tests with resolution of screenshots - 1050x938
I tried to log screen.width/screen.height, and looks like screen resolution on CI - 1280x1024

On before:browser:launch I see chrome arg --start-maximized, seems it does not work at all?

I'm totally new to xvfb and how cypress work with it. Please point me to the right direction, is there any way to get desired resolution?

[ykravv]

Hey @brian-mann

We'd need to support setting xvfb viewport via the command-line

Is there any possibility to do that?

[jennifer-shehane]

You can pass a -screen option to xvfb-run specifying the resolution. See XVFB docs

Code where Cypress passes 1280x1024x8 option: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/scripts/run.js#L55

−screen screennum WxHxD
This option creates screen screennum and sets its width, height, and depth to W, H, and D respectively. By default, only screen 0 exists and has the dimensions 1280x1024x8.

[ykravv]

Thanks @jennifer-shehane

I'm trying to get it working with docker image cypress/browsers:chrome67-ff57, in my docker file I have the following lines:

RUN Xvfb :1 -screen 0 '1600x1024x16'  &
RUN export DISPLAY=:1

Is that enough for cypress to pick up the display or am I missing something?
Update: Looks it does not work, getting cy.screenshot failing on timeout

[jennifer-shehane]

@YevheniiKravchenko Sorry, I was addressing the core changes needed for cypress to fix this issue.

To pass the options to chrome, you will have to pass the args slightly differently. Example of plugins file:

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--cast-initial-screen-width=1600')
      args.push('--cast-initial-screen-height=900')

      return args
    }

    if (browser.name === 'electron') {
      args.width = 1600
      args.height = 900

      return args
    }
  })
}

I didn't try this locally, so let me know if this works! Be sure to check our browser launch api docs.

[mgrybyk]

any updates on how to set screen size to desired value instead of hardcoded 1280x720 ?

[AndreVirtimo]

I had the same problem. Our CI server didn't made the screenshots in the right resolution.

I fixed it by changed the https://github.com/cypress-io/cypress/blob/develop/cli/lib/exec/xvfb.js and added xvfb_args. Here is the result: https://gist.github.com/AndreVirtimo/7bd2892478af2ea06ea7445319fbfef1#file-xvfb-js-L12-L16

I'm using hard coded FullHD resolution, but there should be used the viewPort configuration from cypress. I don't know how to get access to the configuration at this point.

The result are screenshots with the resolution 1919x1080. I don't now why the is a pixel missing. But for me this is ok at this point.

[lukeapage]

On windows I've had some success running this in headless mode before cypress:

https://gallery.technet.microsoft.com/scriptcenter/2a631d72-206d-4036-a3f2-2e150f297515

[infctr]

@jennifer-shehane Setting width and height for headless electron in docker container produces screenshots with correct size (1919x1080 actually), however I seem to get a video recording error.

// config
module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'electron') {
      args.width = 1920;
      args.height = 1080;

      return args;
    }
  })
}

Warning: We failed to record the video.

This error will not alter the exit code.

Error: ffmpeg exited with code 1: Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

    at ChildProcess.<anonymous> (/root/.cache/Cypress/3.2.0/Cypress/resources/app/packages/server/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
    at emitTwo (events.js:125:13)
    at ChildProcess.emit (events.js:213:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

Upd:
This seems related #3491

Upd2:
My current workaround is to add an extra 1px to Xvfb display setup

#!/bin/sh
Xvfb :1 -screen 0 1921x1080x24 >/dev/null 2>&1 &
export DISPLAY=:1.0

yarn nps test.cy.ci

[timmydoza]

@infctr What docker image are you using? I can't get the args.width = 1400 trick to work. Also where are you changing the settings for xvfb? Is it in the dockerfile, or is the script run somewhere else?

[infctr]

@timmydoza I've extended the image from the official repo https://github.com/cypress-io/cypress-docker-images/blob/master/browsers/chrome69/Dockerfile, added latest yarn and dependencies needed for my project.

The script above runs on CI as a job, sets xvfb display settings and starts the tests. You can run it as an entrypoint for you docker container. Nothing really fancy here

[timmydoza]

Perfect - got it working. Thanks! Couldn't get it working with chrome, but it works with electron and the base:10 image.

[tonilopezmr]

@AndreVirtimo And how did you run that on the CI?

I have tried to add these arguments but it still failing: https://github.com/Karumi/LitElementPlayground/runs/268134245

I'm copying xvfb.js file with the arguments into the node_modules xvfb.js running this script https://github.com/Karumi/LitElementPlayground/blob/add-cypress/cypress/scripts/xvfbargs.js.

And this is the cypress dashboard: https://dashboard.cypress.io/#/projects/144fs5/runs

Edit:

Here is on travis: configuration travis file

Same error on dashboard: https://dashboard.cypress.io/#/projects/144fs5/runs/14/specs

Now it seems if I add on travis the service: xvfb now the size is lower (1023x767)

Edit2:

It worked on travis!!! Editing the viewport for electron, and executing xvfb --screen adding one more pixel in width and height worked!!!!!

[ispal]

@ispal in headless ci in Windows you need to change the screen resolution. I do this : #2102 (comment)

Yeah, maybe that or I might give this a go: https://docs.microsoft.com/en-us/powershell/module/servercore/set-displayresolution?view=win10-ps

[sebinsua]

Since posting my last comment, I upgraded to Cypress 3.6 with headed Chrome 76 within a Docker container.

After doing this the maximum width of the browser in headless mode ended up being 1050px. That is despite the viewportWidth being set to 1280px. It appears to be clipped to this value, since by using Xvfb as others did above I could only force the width to be a number below 1050. I'm not sure why this is the case.

I've noticed that the number 1050 does appear in a number of issues where people are using --browser chrome in a Docker container, so I don't think I'm the only person to get this.

For now, the quick fix seems to be setting the viewportWidth to 1050px, since this value is obeyed by both headless and headed mode.

[WolfgangFellger]

@sebinsua We experienced the same issue. Passing --window-size seems to do the trick (at least in cypress 3.7) without the other shenanigans mentioned in this thread:

on('before:browser:launch', (browser = {}, args) => {
	if (browser.name === 'chrome') {
		args.push('--window-size=1920,1080')
		return args
	}
})

[memee]

You can pass a -screen option to xvfb-run specifying the resolution. See XVFB docs

Code where Cypress passes 1280x1024x8 option: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/scripts/run.js#L55

−screen screennum WxHxD
This option creates screen screennum and sets its width, height, and depth to W, H, and D respectively. By default, only screen 0 exists and has the dimensions 1280x1024x8.

@jennifer-shehane I would rather not modify cypress code. Is there a recommended way to pass custom resolution to Xvfb with cypress? In plugins maybe. Besides, I'm using image: cypress/browsers:chrome69 in gitlab-ci and cypress-plugin-snapshot. I found the similar issue there: meinaart/cypress-plugin-snapshots#71

[warpech]

I got lucky with resolution 1980x1080 in headless Chrome in Docker, without messing with xvfb.

I used the Docker image cypress/included version 3.8.3. My config:

1. In cypress.json, set:

   "viewportWidth": 1920, 
   "viewportHeight": 1080

2. In cypress/plugins/index.js, set:

   on('before:browser:launch', (browser = {}, args) => {
     if (browser.name === 'chrome') {
       args.push('--window-size=1920,1080');
       return args;
     }
   });

3. In the tests, set cy.viewport(1920, 1080);
4. My Docker-compose YML file that I use for running Cypress:

   version: '3.5'
   services:
     cypress:
       image: "cypress/included:3.8.3"
       environment:
         - CYPRESS_baseUrl=http://host.docker.internal:8080
         - CYPRESS_cmd
       working_dir: /app
       volumes:
         - ../:/app
       command: --browser chrome --headless ${CYPRESS_cmd}

${CYPRESS_cmd}` is used to pass any additional command options, such as the name of the test file to run.

---

As mentioned in the beginning, this works with the resolution 1920x1080. At first, I tried 3840x2160 but cypress run froze at executing the first test. Later, I tried 2560x1440, but cypress run was still freezing at some tests 50% of the time. I settled on 1920x1080 and did not investigate further.

[RichieRunner]

I got lucky with resolution 1980x1080 in headless Chrome in Docker, without messing with xvfb.

I used the Docker image cypress/included version 3.8.3. My config:

1. In cypress.json, set:

   "viewportWidth": 1920, 
   "viewportHeight": 1080

2. In cypress/plugins/index.js, set:

   on('before:browser:launch', (browser = {}, args) => {
     if (browser.name === 'chrome') {
       args.push('--window-size=1920,1080');
       return args;
     }
   });

3. In the tests, set cy.viewport(1920, 1080);
4. My Docker-compose YML file that I use for running Cypress:

   version: '3.5'
   services:
     cypress:
       image: "cypress/included:3.8.3"
       environment:
         - CYPRESS_baseUrl=http://host.docker.internal:8080
         - CYPRESS_cmd
       working_dir: /app
       volumes:
         - ../:/app
       command: --browser chrome --headless ${CYPRESS_cmd}

${CYPRESS_cmd}` is used to pass any additional command options, such as the name of the test file to run.

As mentioned in the beginning, this works with the resolution 1920x1080. At first, I tried 3840x2160 but cypress run froze at executing the first test. Later, I tried 2560x1440, but cypress run was still freezing at some tests 50% of the time. I settled on 1920x1080 and did not investigate further.

---

Still doesn't work for me. There are still differences with just running run and open locally. This is just silly.

[RichieRunner]

Yeah, for that reason I never create screenshots locally. I only use cypress gui locally for debugging without screenshots, or docker-compose to run Cypress in Docker with screenshots. That setup gives me the same screenshots in Windows, Mac, Linux and CI (GitHub Actions).

oh, so do you mean create the baseline images FIRST through your CI build pipeline? Interesting. Then where/how would you find it to commit into into your source control?

[laurogripa]

Possibly related: electron#21379: Screen dimensions are off by one pixel in offscreen mode

[lnbxyz]

Yeah, for that reason I never create screenshots locally. I only use cypress gui locally for debugging without screenshots, or docker-compose to run Cypress in Docker with screenshots. That setup gives me the same screenshots in Windows, Mac, Linux and CI (GitHub Actions).

oh, so do you mean create the baseline images FIRST through your CI build pipeline? Interesting. Then where/how would you find it to commit into into your source control?

Not necessarily take the screenshots in the CI pipeline, but rather use the same docker image in CI and locally to generate the base screenshots.

[iamjoncannon]

Wanted to update the @warpech solution above for headless Chrome. I spent some time on this yesterday.

Set up is Cypress 4.5 with cypress-image-snapshot in Docker (not using a cypress pre-built image).

I am running VRTs across viewports, and ran into this bug-- the tests were all coming out at 1280px and below, even with the viewport set at higher widths.

The following works at viewports above 1920--

cypress.json

{
    "viewportWidth": 2560, 
    "viewportHeight": 1080
}

cypress/plugins/index.js

on('before:browser:launch', (browser = {}, launchOptions) => {

    if (browser.name === 'chrome') {
      launchOptions.args.push('--disable-dev-shm-usage')
      launchOptions.args.push('--window-size=2560,1080')
      return launchOptions
    }
  })

When I tried @warpech solution initially the suite froze after taking the 1920 shot, but "disable-dev-shm-usage" ended up resolving the issue-- the suite loads at 2560 and reduces the screensize correctly for each it block. (Lower viewports are less memory intensive, which is why running at default 1280 works even without the shm flag)

@jennifer-shehane It might make sense to add this information here-- https://docs.cypress.io/api/plugins/browser-launch-api.html#Set-screen-size-when-running-headless-Chrome-- and/or here-- https://docs.cypress.io/guides/guides/continuous-integration.html#In-Docker -- and make it a bit more explicit that this is the solution to this issue.

The basic idea is-- headless Chrome in CI needs to load at the max viewport width the test will require, and from there it will resize properly during the run. Disabling shm is not only recommended but likely required for higher viewports.

[FDiskas]

Got it working by following the docs https://docs.cypress.io/api/plugins/browser-launch-api.html#Set-screen-size-when-running-headless

    if (browser.name === 'electron') {
      // fullPage screenshot size is 768x1024
      launchOptions.preferences.width = 768;
      launchOptions.preferences.height = 1024;
      launchOptions.preferences.frame = false;
      launchOptions.preferences.useContentSize = true;
    }

[jennifer-shehane]

I'm going to close this as a duplicate of #587 "Make screenshot/video size configurable" because overall, I think this entire thread demonstrates a lack of a straightforward way to explicitly set the size you want screenshots (and videos) to be taken at.

I've noted all of the feedback within this thread for reference when we move forward with implementing a solution to this. Thanks everyone. This issue will be left open for comments so that workarounds can continue to be discussed openly.

[andrewalc]

Got it working by following the docs https://docs.cypress.io/api/plugins/browser-launch-api.html#Set-screen-size-when-running-headless

    if (browser.name === 'electron') {
      // fullPage screenshot size is 768x1024
      launchOptions.preferences.width = 768;
      launchOptions.preferences.height = 1024;
      launchOptions.preferences.frame = false;
      launchOptions.preferences.useContentSize = true;
    }

The documentation link solved my issue. I went down a rabbit hole of issues in this repo to find this. Just want to leave some notes on how i got here and give some thoughts / suggest some documentation edits.

My initial issue was my headless screenshots were being scaled to a smaller resolution. This caused the charts i was screenshotting to be cut in half, which would not do for my image-regression tests. My first assumption was cy.screenshot's scale option had something to do with it. When i gave up on that I assumed viewportWidth and viewportHeight in cypress.json would affect the headless width and height, which lead me to try and use cy.viewport to change this to no avail. I was starting to get concerned until i found this comment, seeing the issues surrounding screenshots not behaving in headless mode still open in this repositiory was leading me to abandon this effort.

It would be incredibly helpful if this section was linked on the cy.viewport or cy.screenshot docs page. clarifying that scaling issues with screenshots in headless mode is solved by this specific plugin config where you set your desired resolution, or a mention of how cy.viewport in headless mode doesn't behave correctly and to get your screenshots to look the way you want you need to edit the plugin config. I basically just replaced the resolution values with the values i had defined in viewportWidth and viewportHeight already.

The fact that viewportWidth and viewportHeight does not affect headless viewport height and width despite the docs saying it is global seems incredibly misleading to me. The mention that these two variables are Global at the top of the page leads me to expect this should affect the viewport everywhere, including headless mode. There is no mention that these two variables do not affect headless mode here either. If the intent is to not affect headless mode and explicitly define it through a plugin, then perhaps runnerViewportWidth and runnerViewportHeight would describe this better. Otherwise I would really hope this would set the default viewport resolution everywhere, whether its in the runner or headless.

[TeemuKoivisto]

I'm not sure if this is related but as I'm using screenshot snapshots https://github.com/meinaart/cypress-plugin-snapshots my snapshots are failing when I switch from headed to headless Chrome. I tried all the options I could find until I realized that in headed mode Chrome starts windowed (or kiosked - dont know the term) while in headless it always starts full-screen. Thus adding launchOptions.args.push('--start-fullscreen') to my config seemed to have solved that discrepancy.

Just seems super weird there would be a problem like this. I'll see if I can update my config to start always windowed as debugging in full-screen is quite annoying. But in any case, hope this helps someone.

[FDiskas]

In testcafe case we pass arguments to chrome executable.

testcafe -q "chrome:headless '--window-size=1366,950'"

[jennifer-shehane]

Yes, headed Chrome has a strange way of setting the screen size. We definitely recommend using the same headed/headless value when comparing snapshots. Hopefully we can standardize the Chrome headed size in the future.