Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress UI locks up during cypress run in CI (recognizable in video recording, the UI is less responsive and/or completely frozen) #4722

Closed
jennifer-shehane opened this issue Jul 16, 2019 · 13 comments

Comments

@jennifer-shehane
Copy link
Member

@jennifer-shehane jennifer-shehane commented Jul 16, 2019

Current behavior:

As per the following comments, the UI appears to freeze when reviewing video taken during cypress run. There is only activity recorded for the first few seconds.

Example videos https://52855-7910045-gh.circle-artifacts.com/0/root/project/cypress/videos/main-page.spec.js.mp4, https://53812-7910045-gh.circle-artifacts.com/0/root/project/cypress/videos/main-page.spec.js.mp4. But I think that these "frozen" videos are a separate problem. Original problem described in this issue is about incomplete videos.

Desired behavior:

The UI/video recording should not freeze and capture the test run like usual when run during cypress open

Steps to reproduce: (app code and test code)

??? I'd love if anyone could respond with the Versions below and ideally steps to reproduce this. 🙏

Versions

Cypress: <= 3.3.1
Browser: Electron 61
OS: Linux, Debian 8-1 + ??
CI: Jenkins + ??

@jennifer-shehane

This comment has been minimized.

Copy link
Member Author

@jennifer-shehane jennifer-shehane commented Jul 16, 2019

I would appreciate if everyone with this issue would update to our latest version of Cypress - we made several improvements to the performance of the Test Runner that may have helped with the freezing. Please post any results here.

cc @platan @pisti2010 @maryhipp

@cooleiwhistles

This comment has been minimized.

Copy link

@cooleiwhistles cooleiwhistles commented Jul 16, 2019

Hi, I am on latest version of Cypress (3.4.0) and running tests through Jenkins CI on a Linux machine. I am seeing issues with the test recording freezing, and then jumping ahead toward the end. When testing locally the recordings seem fine. Once we added the dockerfile and split the tests in parallel, we began seeing the issue.

@maximkoshelenko

This comment has been minimized.

Copy link

@maximkoshelenko maximkoshelenko commented Jul 16, 2019

Have the same issue. Cypress 3.4.0, Gitlab Pipeline runner, cypress/browsers:node12.6.0-chrome75 image, npx cypress run script. Image record only a few seconds and freeze after.

@platan

This comment has been minimized.

@platan

This comment has been minimized.

Copy link

@platan platan commented Jul 16, 2019

@bahmutov

This comment has been minimized.

Copy link
Contributor

@bahmutov bahmutov commented Jul 26, 2019

I have done some experiments, trying to recreate a frozen video

  • using https://github.com/bahmutov/demo-docker-cypress-included I have added --memory and --cpu restrictions to the Docker container as explained in https://docs.docker.com/config/containers/resource_constraints/
  • I can monitor CPU and memory usage via docker stats <container id>
  • when restricting memory, the test runner either runs or not, and the ffmpeg either works no gets killed when it hits the memory limit. Still, it runs reliably under 370 megabytes.
    NOTE: the 370MB worked for this small web site, that did not load much during tests. If the web application were larger, and the test were longer, I am sure the browser and Cypress would need more memory to finish and not crash.
  • when restricting CPU share, the entire test becomes slower, the video becomes longer. Finally, for the following parameter on my Mac Book Pro I could see a video with freezes. There was part of the middle missing and entire second half
--cpus=0.2

Note that I have enabled full memory, so the CPU is the deciding factor

Other debugging notes:

  • you can see the amount of memory each second by using CYPRESS_MEMORY=1 environment variable

So I think the real culprit is the CPU resources available on CI during test runs. Remember - it is not just Cypress test runner, there might be other services running too: the server, the bundler, databases, etc.

Recommendation:

try running CI tests on a more powerful box. If the box is already powerful, try running fewer concurrent jobs.

@WillsB3

This comment has been minimized.

Copy link

@WillsB3 WillsB3 commented Jul 26, 2019

Recommendation: try running CI tests on a more powerful box. If the box is already powerful, try running fewer concurrent jobs.

As far as I'm aware we don't have any control over the assigned hardware resources with GitLab CI, so I don't think I can personally test that suggestion unfortunately.

Additionally, with our CI pipeline, the E2E stage is the only job running by that point in the pipeline, so reducing concurrent jobs also isn't something I can try.

How do I tell Cypress to record videos when running locally? I tried the --record flag but it wants a key for uploading the video to the Dashboard however I just want the recording stored locally?

@bahmutov

This comment has been minimized.

Copy link
Contributor

@bahmutov bahmutov commented Jul 26, 2019

  • are you using shared GitLab CI runners? Because they are probably underpowered, I would suggest running your own private runners that you do have control over, or even trying CircleCI or SemaphoreCI - we never had problems with videos there, and we are not even using anything but the default boxes.
  • Cypress test runner records video locally by default when you do cypress run. Unless someone has set video: false via configuration options https://on.cypress.io/configuration
@WillsB3

This comment has been minimized.

Copy link

@WillsB3 WillsB3 commented Jul 26, 2019

are you using shared GitLab CI runners? Because they are probably underpowered, I would suggest running your own private runners that you do have control over, or even trying CircleCI or SemaphoreCI - we never had problems with videos there, and we are not even using anything but the default boxes.

Yeah - we're using the shared runners - moving to a different CI platform is not an option for us.

Cypress test runner records video locally by default when you do cypress run. Unless someone has set video: false via configuration options https://on.cypress.io/configuration

Ah - ok, thanks. I'll try running some tests locally too

@dotexe0

This comment has been minimized.

Copy link

@dotexe0 dotexe0 commented Aug 27, 2019

I'm experiencing a similar issue on "cypress": "3.4.1". We use docker-compose in Google Cloud Build to orchestrate e2e testing with some micro-services. I can trigger the e2e tests locally with docker-compose up and the videos get uploaded to the dashboard with no missing frames, but when I pass it through the CI/CD pipeline, the videos in the dashboard freeze after about 3 seconds. The tests always pass and take about the right amount of time to complete even when frozen.

Cypress Dockerfile:

FROM cypress/base:10
WORKDIR /app

COPY package.json .
COPY package-lock.json .

ENV CI=1
RUN npm ci
RUN npx cypress verify

Cypress package.json:

{
  "name": "e2e",
  "version": "1.0.0",
  "description": "Cypress tests",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "license": "ISC",
  "private": true,
  "devDependencies": {
    "cypress": "3.4.1",
    "faker": "^4.1.0"
  }
}

docker-compose:

version: '3'
services:
  web:
    build:
      context: .
      dockerfile: ./Dockerfile 
    ports:
      - 3000
    env_file:
      - .env.development

###--- //  ....microservices.... // ---###

  e2e:
    image: cypress
    build: ./e2e
    container_name: cypress
    environment:
      - CYPRESS_baseUrl=http://web:3000
    command: npx cypress run --group integrations --env ENV=production --record --key #### --parallel --ci-build-id ${BUILD_ID}
    volumes:
      - ./e2e/cypress:/app/cypress
      - ./e2e/cypress.json:/app/cypress.json

cypress.json:

{
  "projectId": "####",
  "baseUrl": "http://localhost:3000",
  "chromeWebSecurity": false,
  "hosts": {
    "https://dev-####.oktapreview.com/api/v1/authn": "127.0.0.1"
  },
  ...
}
@jennifer-shehane jennifer-shehane changed the title UI during video recording is completely frozen Cypress UI locks up during cypress run in CI (recognizable in video recording, the UI is less responsive and/or completely frozen) Aug 29, 2019
@bahmutov

This comment has been minimized.

Copy link
Contributor

@bahmutov bahmutov commented Aug 29, 2019

@dotexe0, unfortunately, this is a problem with underpowered CI container that prevents the video from being captured without gaps or freezing. Try switching to a more powerful container if possible or even to a different CI provider. For example we never have frozen video on CircleCI machines, yet I see freezing video quite often on GitLab CI.

@bahmutov

This comment has been minimized.

Copy link
Contributor

@bahmutov bahmutov commented Aug 29, 2019

We will close this issue since this is due to the underpowered CPU available to Cypress in the CI container. I will think how to detect the frozen video happening and give a warning, have opened #5061

@archfz

This comment has been minimized.

Copy link

@archfz archfz commented Jan 20, 2020

I am also experiencing this. v3.8.1; browser electron 78. Isn't there any way for the ui to wait for the recording ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.