Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR:gpu_memory_buffer_support_x11 on GitHub Ubuntu #25357

Closed
MikeMcC399 opened this issue Jan 4, 2023 · 33 comments
Closed

ERROR:gpu_memory_buffer_support_x11 on GitHub Ubuntu #25357

MikeMcC399 opened this issue Jan 4, 2023 · 33 comments
Labels
E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team.

Comments

@MikeMcC399
Copy link
Contributor

MikeMcC399 commented Jan 4, 2023

Current behavior

When Cypress is run using a GitHub Ubuntu runner, an error message related to graphics is shown before Cypress shows "Run Starting".

The error message contains the following texts:

"ERROR:gpu_memory_buffer_support_x11.cc(44)"
"dri3 extension not supported."

for example:

[1866:0212/100011.310930:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

The error message is output independently of the browser selected, e.g. Electron, Chrome and Firefox on a GitHub runner where debug logs show cypress:cli Starting Xvfb.

No error messages are shown in Ubuntu 22.04 running under VMware Workstation 17 Pro, nor using Ubuntu 22.04 running in Microsoft WSL2 under Windows 11 Pro. In these cases Xvfb is not started (debug log shows cypress:cli Cypress will not spawn own Xvfb).

Desired behavior

If the graphics error listed above does not affect the running of Cypress then their output should be suppressed. The messages can otherwise be misleading for troubleshooting.

The error message is similar to others which are classified as "garbage warnings" in https://github.com/cypress-io/cypress/blob/develop/cli/lib/exec/spawn.js and which are filtered out by that module.

Test code to reproduce

Cypress Version

First reported on Cypress 12.5.1
Issue continues to be reproducible on 13.1.0

Node version

v16.16.0 => v20.5.0

Operating System

Ubuntu on GitHub: ubuntu-20.04 and ubuntu-22.04

Debug Logs

See https://github.com/MikeMcC399/gpu-error-test/actions/workflows/gpu-test-base.yml

Other

Related issues are:

Workaround

If Cypress tests are failing and the cause is unclear it may be useful to rule out any gpu error by making sure that the gpu error described here does not occur.

To prevent the gpu error, add

env:
  ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu'

to the GitHub workflow and use a browser other than the default Electron browser.

Edit: reworked issue

@astone123
Copy link
Contributor

I also see this when running Ubuntu in Github. Routing this to the e2e team to let them determine how they want to handle this.

@ajones270
Copy link

I also see this when running Ubuntu 22.04 running via Azure Pipelines

@lmiller1990
Copy link
Contributor

Just to clarify and priotize, is this blocking your test run, or just a quality of life fix? I have seen this warning but it has not prevented running my tests. If you are finding this a blocker, please share, we can prioritize accordingly.

@MikeMcC399
Copy link
Contributor Author

@lmiller1990

Just to clarify and priotize, is this blocking your test run, or just a quality of life fix? I have seen this warning but it has not prevented running my tests. If you are finding this a blocker, please share, we can prioritize accordingly.

I have never seen it blocking a test, I have however seen the error quoted by multiple people as being the reason for their test failing, since it is often the last error message which is thrown before the point where users are expecting Cypress to run.

Addressing this issue would help users with an error condition to be able to focus on what the real error might be. So it's a bit more than just a "quality of life" bug. Also it is easily reproducible and it shows up on everybody's test runs.

So I do believe you ought to be putting some priority on fixing it.

@rotexhawk
Copy link

rotexhawk commented Jan 31, 2023

Agree with the comment ^. This error randomly fails our pipeline and it is time consuming the run the steps again.

@lmiller1990
Copy link
Contributor

I agree this is pretty confusing, I have a feeling pipelines are failing for other reasons, which we are either incorrect suppressing or not surfacing. Since this is the last warning people see, they assume it's related (could be, but might be a red herring).

I see (at least) two issues to solve:

  1. my pipeline fails and I don't know why, but I see this warning and googled and landed here - what can I do
  2. this warning is meaningless/red herring, let's get rid of it

This issue is for suppressing the warning (point 2), let's track that here. If you have a failing pipeline and you can reproduce it in a minimal fashion and believe it's related (point 1), please make a new issue and reference this one, we can help fix it, and hopefully add a better error handling around whatever issue is found.

@MikeMcC399
Copy link
Contributor Author

Thank you @lmiller1990 for expanding the description above ^ !

My issue is only about the red herring aspect.

It would be great if anybody else, like @rotexhawk, who says that the issue is causing their pipeline to fail could open a new issue with minimal repro steps.


Do you still need me to keep my example repo https://github.com/MikeMcC399/gpu-error-test available or can I delete it now?

There are plenty of examples of this error showing up in the cypress-io/github-action repo that can be seen when viewing the runs of https://github.com/cypress-io/github-action/blob/master/.github/workflows/example-basic.yml

Search under https://github.com/cypress-io/github-action/actions/workflows/example-basic.yml?query=branch%3Amaster for a selection of logs.

@lmiller1990
Copy link
Contributor

I think you can delete it, the repro is basically: run Cypress on Ubuntu on CI. Thanks!

@MikeMcC399
Copy link
Contributor Author

@lmiller1990

I think you can delete it, the repro is basically: run Cypress on Ubuntu on CI. Thanks!

I have deleted it. As you say, it is easy to reproduce.

I revisited it and I find that with Cypress 12.2.0 and ubuntu-20.04 or ubuntu-22.04 the error:

libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

is no longer shown and I assume this is due to updates in the GitHub runners.

With Cypress 12.5.1 the error message:

[1902:0209/105331.818311:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

is however still being displayed, so this issue remains valid.

@MikeMcC399
Copy link
Contributor Author

@MikeMcC399
Copy link
Contributor Author

@MikeMcC399
Copy link
Contributor Author

@MikeMcC399
Copy link
Contributor Author

MikeMcC399 commented Feb 12, 2023

Workaround

Add

env:
  ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu'

to the GitHub workflow. The result is that the error message

[1866:0212/100011.310930:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

disappears for any selected browser except Electron. For Electron the error message is output at a later stage.

A partial explanation is that with no environment variable ELECTRON_EXTRA_LAUNCH_ARGS defined, the error message is output at a pre-browser selection stage when the built-in Electron browser is running. Adding the environment variable stops Electron trying to use the gpu at this point.

According to Configure browser environment, which shows how to pass an environment variable when a browser is launched, it says "This option is not supported when targeting Electron", so presumably this is the reason that ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu' is having no effect later when Electron has been specifically selected to run as browser.

I reinstated a repro repo at https://github.com/MikeMcC399/gpu-error-test

The remaining error in the electron browser case with full debug logging enabled is shown at:

https://github.com/MikeMcC399/gpu-error-test/actions/runs/4155772179/jobs/7189104496#step:3:1110

  cypress:server:browsers:electron launching browser window to url: http://localhost:45827/__/#/specs/runner?file=cypress/e2e/spec.cy.js +1ms
  cypress:server:browsers:electron debugger attached +14ms
  cypress:server:browsers:electron clearing cache +0ms
[1838:0212/100028.728185:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

@stychu
Copy link

stychu commented Feb 16, 2023

#23641 (comment)

@lmiller1990
Copy link
Contributor

I will post in #23641

@mysticdevx
Copy link

I am getting the same error in GitLab runners with the following env.
docker image: cypress/base:14.17.3,
cypress: v12.7.0
browser: both in Chrome 108 and Electron 106

@MikeMcC399
Copy link
Contributor Author

@mysticdevx

I am getting the same error in GitLab runners with the following env.
...
cypress: v12.7.0
browser: both in Chrome 108 and Electron 106

That corresponds to my original post that reported the issue on Cypress 12.5.1 and confirmed that it is still reproducible on Cypress 12.7.0. You can use the workaround in the original post for Chrome, but it doesn't help with Electron.

@bilarallen
Copy link

Hello, I'm also encountering this when running parallel test in Jenkins

Before the execution run starts, this error [1993:0313/103123.349474:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported. displays on every parallel stage
image

And during the run this error [2162:0313/103202.872609:ERROR:gl_utils.cc(319)] [.WebGL-0x1590041ac600]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels randomly display
image

Here's additional info:
Cypress version: 12.7.0
CI: Jenkins
Docker image: cypress/base:14.16.0
Browser: Electron 106

@MikeMcC399
Copy link
Contributor Author

MikeMcC399 commented Mar 13, 2023

@bilarallen

And during the run this error [2162:0313/103202.872609:ERROR:gl_utils.cc(319)] [.WebGL-0x1590041ac600]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels randomly display

  • If you think the above error message needs to be investigated, then I suggest you open a new issue, since this most likely has a different root cause. You might also want to Google this error message.

  • In Errors on execution on Azure Pipeline agent #23641 (comment) there is a comment from @lmiller1990 asking if anybody had run with DEBUG=cypress:* for your issue. It would certainly make sense for somebody encountering this error to open a separate issue here.

@nagash77 nagash77 added E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team. labels Apr 19, 2023
@andrii-lundiak
Copy link

andrii-lundiak commented Jul 21, 2023

I do confirm, that setting

env:
  ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu'

does NOT help to avoid :

[2354:0721/173502.928020:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported

I updated my build.yml file which is used by Azure DevOps CI setup. No Docker directly per se.

Agent name: 'Hosted Agent'
Agent machine name: 'fv-az***-***'
Current agent version: '3.220.5'
Operating System
Ubuntu
22.04.2
LTS
Runner Image
Image: ubuntu-22.04
Version: 20230716.1.0
Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230716.1/images/linux/Ubuntu2204-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230716.1
Runner Image Provisioner
2.0.238.1
Current image version: '20230716.1.0'
Agent running as: 'vsts'

My Cypress code is running on top of NodeJS 18.x and Cypress itself is 12.17.2 which has been assuming that MAY help to get rid some Electron issues. CHANGELOG here: https://docs.cypress.io/guides/references/changelog#12-17-2

UPD. I know that env variable suggest that something with Electron, but I also experience this dri3 extension not supported when Chrome browser is sued on CI/Ubuntu:
dri3_warning_via_Chrome_

And here is screenshot for Electron run:
dri3_warning_via_Electron

And BTW, when I use DEBUG=cypress:* enviromental variable I have then such log:

image

Meaning that it doesn't hint me much of what the reason could be.

@andrii-lundiak
Copy link

andrii-lundiak commented Jul 25, 2023

@MikeMcC399, @ricardo-dematos, @tgreen7 do you still have this issue? I mean gpu_memory_buffer_support_x11 on or dri3 extension not supported issue.

@andrii-lundiak
Copy link

andrii-lundiak commented Jul 25, 2023

And actually, maybe there is NO need "listening" on CI?
DevTools listening on ws://127.0.0.1:60830/devtools/browser/c81bcbfe-c257-4d31-b45b-6e32e9d0eb20 (I assume it's Electron/Chrome devtool watch/livereload/hotreload feature)
How can I disable this to be 1) used, 2) thrown?

@MikeMcC399
Copy link
Contributor Author

@andrii-lundiak

@andrii-lundiak
Copy link

andrii-lundiak commented Jul 25, 2023

@MikeMcC399 I know, but I assumed that it's just after that log entry, and I suspected that maybe it's related (see my above comment w screenshots). But thanks for that another issue. Will research.

@samueljholiver
Copy link

some solution worked? I have same problem with azure devops pipeline with ubuntu 20.04

@lmiller1990
Copy link
Contributor

What is the actual problem? If it's just some text, I don't think there's anything to do - it's just telling you about your environment.

If you really don't like the warning, you can probably disable it. I googled for about 10 seconds and found: seleniumbase/SeleniumBase#1384

Seems a common issue in many other tools, eg selenium seleniumbase/SeleniumBase#1384 / PW microsoft/playwright#15907

Anything for us to do here? I don't think so, Cypress should not be swallowing warnings - if anything, that would make debugging more confusing.

@MikeMcC399
Copy link
Contributor Author

@lmiller1990

If you really don't like the warning, you can probably disable it. I googled for about 10 seconds and found: seleniumbase/SeleniumBase#1384

  • I'm now a little confused, as the link you quoted (WebGL performance issue -> "GPU stall due to ReadPixels" seleniumbase/SeleniumBase#1384) seemed to be about a different error message, and the original post here already contained a workaround suggestion ELECTRON_EXTRA_LAUNCH_ARGS: '--disable-gpu'.

  • If there is no bug fix possible for Cypress to stop the error message being output, then I would suggest adding it to the documentation on https://docs.cypress.io/ with an explanation about what the error message means, why it occurs and how to prevent it occurring on an individual basis.

  • Unfortunately, although I have been able to reproduce it together with the workaround on https://github.com/MikeMcC399/gpu-error-test/actions, my understanding is not deep enough to be able to document it further with confidence.

@MikeMcC399
Copy link
Contributor Author

MikeMcC399 commented Aug 27, 2023

  • The error message only seems to be displayed when Cypress pulls in Xvfb (virtual framebuffer X server for X Version 11) in a virtual environment where there is otherwise no X11 server available.

  • If Cypress starts Xvfb even when running with Chrome Headless #19868 were to be resolved, then it would probably stop the error message from appearing for at least Chromium type browsers.

Related issues

Related documentation

Continuous integration > Introduction > Running headless tests without Xvfb

@MikeMcC399
Copy link
Contributor Author

Suggestion

Add the error message

[1866:0212/100011.310930:ERROR:gpu_memory_buffer_support_x11.cc(44)] dri3 extension not supported.

to the filtered-out Xvfb "garbage warnings" in

https://github.com/cypress-io/cypress/blob/develop/cli/lib/exec/spawn.js

The error message occurs on GitHub Ubuntu runners when Cypress spawns the Xvfb driver (virtual framebuffer X server for X Version 11).
Cypress does not spawn Xvfb on Windows nor on macOS runners.

As far as I was able to research this topic, Xvfb does not include support for DRI (Direct Rendering Infrastructure). The error is always logged when running Cypress on Ubuntu on GitHub, and on some other CI providers, except notably CircleCI. It does not appear to be detrimental to running Cypress tests.

Other similar garbage errors are already filtered out:

const isXlibOrLibudevRe = /^(?:Xlib|libudev)/
const isHighSierraWarningRe = /\*\*\* WARNING/
const isRenderWorkerRe = /\.RenderWorker-/
// Chromium (which Electron uses) always makes several attempts to connect to the system dbus.
// This works fine in most desktop environments, but in a docker container, there is no dbus service
// and Chromium emits several error lines, similar to these:
// [1957:0406/160550.146820:ERROR:bus.cc(392)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
// [1957:0406/160550.147994:ERROR:bus.cc(392)] Failed to connect to the bus: Address does not contain a colon
// These warnings are absolutely harmless. Failure to connect to dbus means that electron won't be able to access the user's
// credential wallet (none exists in a docker container) and won't show up in the system tray (again, none exists).
// Failure to connect is expected and normal here, but users frequently misidentify these errors as the cause of their problems.
// https://github.com/cypress-io/cypress/issues/19299
const isDbusWarning = /Failed to connect to the bus:/
// Electron began logging these on self-signed certs with 17.0.0-alpha.4.
// Once this is fixed upstream this regex can be removed: https://github.com/electron/electron/issues/34583
// Sample:
// [3801:0606/152837.383892:ERROR:cert_verify_proc_builtin.cc(681)] CertVerifyProcBuiltin for www.googletagmanager.com failed:
// ----- Certificate i=0 (OU=Cypress Proxy Server Certificate,O=Cypress Proxy CA,L=Internet,ST=Internet,C=Internet,CN=www.googletagmanager.com) -----
// ERROR: No matching issuer found
const isCertVerifyProcBuiltin = /(^\[.*ERROR:cert_verify_proc_builtin\.cc|^----- Certificate i=0 \(OU=Cypress Proxy|^ERROR: No matching issuer found$)/
// Electron logs a benign warning about WebSwapCGLLayer on MacOS v12 and Electron v18 due to a naming collision in shared libraries.
// Once this is fixed upstream this regex can be removed: https://github.com/electron/electron/issues/33685
// Sample:
// objc[60540]: Class WebSwapCGLLayer is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebCore.framework/Versions/A/Frameworks/libANGLE-shared.dylib (0x7ffa5a006318) and /{path/to/app}/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libGLESv2.dylib (0x10f8a89c8). One of the two will be used. Which one is undefined.
const isMacOSElectronWebSwapCGLLayerWarning = /^objc\[\d+\]: Class WebSwapCGLLayer is implemented in both.*Which one is undefined\./
const GARBAGE_WARNINGS = [isXlibOrLibudevRe, isHighSierraWarningRe, isRenderWorkerRe, isDbusWarning, isCertVerifyProcBuiltin, isMacOSElectronWebSwapCGLLayerWarning]

@hans2520
Copy link

hans2520 commented Sep 9, 2023

  • The error message only seems to be displayed when Cypress pulls in Xvfb (virtual framebuffer X server for X Version 11) in a virtual environment where there is otherwise no X11 server available.

I can confirm this to be true -- this happens wherever Cypress is run in headerless mode on Ubuntu 20 -- tested with an AWS EC2 that otherwise has gnome GUI enabled and running via command line headerless still reproduces the warning.

Also, the workaround (set ELECTRON_EXTRA_LAUNCH_ARGS='--disable-gpu' in environment prior to launching Cypress) doesn't work at all -- it merely moves the error from appearing before the tests start, to immediately after the tests start as the first line of output.

@MikeMcC399
Copy link
Contributor Author

This issue is no longer reproducible in Cypress 13.2.0.

According to the Changelog:

  • Upgraded Electron from 21.0.0 to 25.8.0, which updates bundled Chromium from 106.0.5249.51 to 114.0.5735.289.
    Additionally, the Node version binary has been upgraded from 16.16.0 to 18.15.0. This does NOT have an impact on the node version you are using with Cypress and is merely an internal update to the repository & shipped binary.
    Addressed in #27715. Addresses #27595.

Presumably this is the reason for the error message no longer appearing.

@hans2520
Copy link

Confirmed issue is fixed in 13.2.0

@AlexanderTunick
Copy link

Confirmed

Confirmed issue is fixed in 13.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team.
Projects
None yet
Development

No branches or pull requests