New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large XHR response objects can exceed the maximum header size. #76

Open
zachgibb opened this Issue Oct 27, 2015 · 12 comments

Comments

10 participants
@zachgibb
Copy link

zachgibb commented Oct 27, 2015

If I respond to route with a very large object, tests waiting for that response will fail with ERR_EMPTY_RESPONSE

cy
  .server()
  .route("POST", /route/, reallyLargeObject).as("willFail")
  .get(".submit").click()
  .wait("@willFail")
@brian-mann

This comment has been minimized.

Copy link
Member

brian-mann commented Oct 29, 2015

Okay this is not a quick or easy fix.

Node sets a constant maximum total header size at 80kb for all headers. There is no way to get around this. We can't toss the data in the request body either, since that would be altering the entire request.

Probably the only way to do this is transfer the data over websockets while the server begins handling the request.

That means managing all kinds of state, but this would work. There are some other features of the XHR refactor that would also introduce websockets, so I'll hold off on implementing this until it can go in with some new architecture.

In the mean time you'll have to reduce the response sizes to less than 80kb, or just use a fixture and not an alias.

So just move reallyLargeObject to a fixture and update your route

cy.route("POST", /route/, "fixture:reallyLargeObject")
@acthp

This comment has been minimized.

Copy link

acthp commented Dec 7, 2017

Trying to find a workaround for this, e.g. manipulating the request, or the response, or the server object. The available hooks fire after the request is sent, and after the response is delivered to the app, so there's no opportunity to influence it. Also haven't found any way to access the server object, e.g. to override a method. Any suggestions? Using a fixture doesn't help, because it's fixed. ;)

@brian-mann

This comment has been minimized.

Copy link
Member

brian-mann commented Dec 7, 2017

If your fixture exceeds the maximum headers you'll need to modify your app code, such as using a conditional if window.Cypress exists, and if so, to use a property on that (on the XHR response callback)

Then ahead of time in your test code, you can use a fixture and set that property on the Cypress object so your app code can pick it up.

@brian-mann

This comment has been minimized.

Copy link
Member

brian-mann commented Dec 7, 2017

The key takeaway here is that Cypress is just javascript and it has native access to everything. You can share properties and data. You can collaborate between Cypress and your application code.

If your application code is "hardwired" to take a specific path, you can just modify it to open a seam to which you can use Cypress to modify that behavior.

@acthp

This comment has been minimized.

Copy link

acthp commented Dec 7, 2017

Just realized I could wrap onreadstatechange during onRequest, to flow the data into the app.

cy.route({
	url: '/api/bookmarks/bookmark*',
	method: 'GET',
	onRequest: xhr => {
		var orsc = xhr.xhr.onreadystatechange;
		xhr.xhr.onreadystatechange = function() {
			if (this.readyState === 4) {
				Object.defineProperty(this, 'response', {
					writable: true
				});
				this.response = content;
			}
			orsc.apply(this, arguments);
		};
	},
	response: 'placeholder',
});

The Cypress console shows response 'placeholder', but otherwise this functions as a workaround.

@brian-mann

This comment has been minimized.

Copy link
Member

brian-mann commented Dec 7, 2017

Yup that would work.

You could also bind to onload instead of onreadystatechange. Would avoid the readyState conditional

@ankri

This comment has been minimized.

Copy link

ankri commented Feb 9, 2018

Thank you for your suggestions.

For guys coming from google. Here is our workaround:

put this in e.g. integration/utils/loadLargeFixture.js

export default function loadLargeFixture(url, response, method = 'GET') {
  return cy.route({
    url,
    method,
    onRequest: xhr => {
      const originalOnLoad = xhr.xhr.onload;
      xhr.xhr.onload = function() {
        Object.defineProperty(this, 'response', {
          writable: true
        });
        this.response = response;
        originalOnLoad.apply(this, xhr);
      };
    }
  });
}

Import the json and the loadLargeFixture function in your test:

import largeFixture from '../fixtures/veryLargeJson.json';
import loadLargeFixture from '../utils/loadLargeFixture';

describe('demo', () => {
  it('should work with large fixture', () => {
    cy.server();
    loadLargeFixture('/api/large', largeFixture).as('data');
    // do stuff
    cy.wait('@data')
    // do stuff
  });
});
@jennifer-shehane

This comment has been minimized.

Copy link
Member

jennifer-shehane commented May 1, 2018

I spent some time stuck on this today. 😢 It wasn't very obvious because I made a change to my fixtures that must have JUST put it over size limit for responses. So working yesterday -> not working today.

The ERR_EMPTY_RESPONSE would not display on each failure either, instead if would just say:

CypressError: The network request for this XHR could not be made. Check your console for the reason.

I was able to inspect the network panel and see that the response had a size of 0 bytes, which clued me in on it being this issue.

screen shot 2018-05-01 at 3 19 02 pm

This only breaks when I generate a bunch of data for an array response (like to check pagination), so would be nice to have fixed.

@tolicodes

This comment has been minimized.

Copy link
Collaborator

tolicodes commented Aug 14, 2018

hah just found a super simple solution:
before I had

cy.fixture(`${endpoint}.json`).as(`fixture${endpoint}`);
cy.route(path, `@fixture${endpoint}.json`)
    .as(endpoint);

now:

cy.route(path, `fixture:${endpoint}.json`)
    .as(endpoint);

Cleaner and it works :)

@Kebie

This comment has been minimized.

Copy link

Kebie commented Oct 25, 2018

This error mixed with the no support for fetch() ruined my afternoon, and made me think I was crazy. Please add note in docs where it is suggested to use cy.fixture aliases when it seems to not be great for large jsons.

Just not using aliases like @tolicodes suggested fixed everything.

@saimonmoore

This comment has been minimized.

Copy link

saimonmoore commented Dec 5, 2018

I just ran into this today! @tolicodes method worked perfectly for me with a 1Mb fixture!

@cookiescrumbs

This comment has been minimized.

Copy link

cookiescrumbs commented Dec 5, 2018

@jennifer-shehane i'm getting the same CypressError: The network request for this XHR could not be made. Check your console for the reason.

Looking at the solution from @brian-mann at the top; I'm confused about the solution and why what I'm doing won't work... I'm not using an alias.

options = {
    method: 'POST',
    status: 200,
    url: '/playlist/search'
}

cy.fixture('playlist/search/playlist-search.dto.mock.json').then((data) => {
    options['response'] = data;
    cy.route(options);
});

Cheers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment