fix: memory leak caused by storing base64 encoded files recieved by CDP Network.requestWillBeSent

[DJSdev]

• Closes Tests causing CI agents to run out of memory on cypress 8.2 & above #17853

User facing changelog

Fixed a memory leak in Chromium-based browsers caused by storing data: url containing base64 encoded files unnecessarily. Fixes #17853

Additional details

Summary

There is a memory leak in the CdpAutomation class that causes Cypress to store data urls as requests in the pendingBrowserPreRequests array.

Background

After updating Cypress to any version higher than 8.2, my e2e suite crashes with the following error after about an hour:

8672:0000371000344000]  3709798 ms: Mark-sweep 1952.5 (2070.7) -> 1946.9 (2071.4) MB, 450.0 / 0.0 ms  (average mu = 0.789, current mu = 0.668) task; scavenge might not succeed
[8672:0000371000344000]  3711538 ms: Mark-sweep 1953.7 (2071.4) -> 1947.8 (2072.4) MB, 350.9 / 0.0 ms  (average mu = 0.793, current mu = 0.798) task; scavenge might not succeed

<--- JS stacktrace --->

[8672:0622/112740.931:ERROR:node_bindings.cc(143)] Fatal error in V8: Reached heap limit Allocation failed - JavaScript heap out of memory

While the tests run, I inspected task manager and I noticed the Cypress server process growing in size until the machine's memory is exhausted causes this issue.

Investigation

I started off by adding code to Cypress to have it dump it's heap memory every 90 seconds.

After 3 minutes of tests, you can see a big difference in the heap sizes between the Cypress 8.2 (first entry) and Cypress 8.1 (second entry)

After inspecting the heap memory in the DevTools console I discovered that Cypress is storing hundreds/thousands of copies of a font file.

This is a font-file used by my application and it's referenced like so:

I discovered in Chrome, when a CSS style sheet uses the url() function in the src field with a base64 encoded file (a data: url), it still triggers Chrome to "send" a network request even though it's not actually going to send one.

Cypress subscribes to the Network.requestWillBeSent event in CDP. When Chrome goes to 'fetch' the base64 encoded file, it sends a requestWillBeSent event to Cypress which stores the url (literally a whole file) in the pendingBrowserPreRequests array. Every time the page is loaded with the CSS, a request is sent adding to the buffer until Cypress eventually crashes.

Solution

The solution I went with is very basic. Simply filtering out data: urls from from the Network.requestWillBeSent event handling logic should clear it up.

Another thought was to have logic clear out stale entries from the array, but that's a lot of work for URLs that shouldn't be stored to begin with.

Steps to test

I can generate a test repo that reproduces this issue if anyone requests it.

I tested this with on version 8.2 and 10.2 and verified the issue existed on both versions. (Currently, my team is on 8.0 where this issue doesn't happen. This issue has stopped us from upgrading for a very long time).

I verified after implementing this fix on 10.2, the memory leak no longer happens and data urls are no longer stored in the pendingBrowserPreRequests array.

How has the user experience changed?

Nothing has changed!

PR Tasks

• Have tests been added/updated?
• Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
• [na] Has a PR for user-facing changes been opened in cypress-documentation?
• [na] Have API changes been updated in the type definitions?

[cypress-bot]

Thanks for taking the time to open a PR!

• Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
• Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

[CLAassistant]

All committers have signed the CLA.

[mjhenkes]

@DJSdev, Thanks so much for this PR. Would it be possible to get a test written for it?

[DJSdev]

I'll do my best to get a test written for it. It's in a weird place.

[BlueWinds]

I'll do my best to get a test written for it. It's in a weird place.

packages/server/test/unit/browsers/cdp_automation_spec.ts has the existing tests for our CDP integration, and has the helper sendDebuggerCommand which ought to make it easy to send a mocked up Network.requestWillBeSent.

I'd be fine with a test that just emits a Network.requestWillBeSentevent and verifies that onBrowserPreRequest isn't called when it's with a data url.

[DJSdev]

Unit tests have been added. I added some bonus tests around onNetworkRequestWillBeSent and onResponseReceived since there wasn't any around these two functions.

existing tests for our CDP integration, and has the helper sendDebuggerCommand which ought to make it easy to send a mocked up Network.requestWillBeSent.

I wasn't exactly sure how the sendDebuggerCommand worked in regards to these tests. Hopefully triggering the events with the stubbed onFn() will suffice. Looking forward to feedback :)

[flotwig]

Nice!

[BlueWinds]

We have been having some flake problems with our automated test suite recently, which we're working on addressing in several other PRs. Given that the currently failing tests were known to be flaky before this branch, I'm going to exercise my judgement and merge this without waiting for a completely green build.

[Chili82]

Is this fix available on 10.2.0 version?

[vaibhavganatra]

@Chili82 No. 10.2 was released 2 days ago and this PR got approved just today.

[Chili82]

Ok, thanks. I guess it will be in the next release.

[BlueWinds]

We're hoping to get 10.2.1 out tomorrow, but it's not ready quite yet.

[Chili82]

Thank you!

[SadiqRahim]

Does this fix also resolves the issue where it closes the test before it kick start with the error "The Test Runner unexpectedly exited via a close event with signal SIGILL"?

[BlueWinds]

Doesn't sound related to this fix or the linked issue.

[cypress-bot]

Released in 10.3.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v10.3.0, please open a new issue.