multipart/form-data doesn't work with large files

[flotwig]

Original comment by @alegout: #4235 (comment)

Since Cypress 3.3.0, some users can't upload files via a <form> to a server if the file size is over ~16kB

[flotwig]

@alegout re: your last comment: #4235 (comment)

That's failing because of the way the test https server is set up, I added a test for multipart/form-data forms and it works once the https server is set up to accept multipart/form-data. Check out #4241 for the new test.

I will try to reproduce with uploading files.

[alegout]

thanks @flotwig i confirm it's passing.
I will give you my logs.

[alegout]

I got tests for user registration, with an avatar or without.
Test registration without avatar is passing, test with avatar got timeout.

https://gist.github.com/alegout/cb8b92125138964ea58d92e472e8e3f6

cypress:server:request received an error making http request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://localhost:8080/en/project/bo/user/register', 
  requestId: 'request9', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 0, 100, 200, 200 ], 
  err: { 
    Error: socket hang up 
      at createHangUpError (_http_client.js:331:15) 
      at TLSSocket.socketOnEnd (_http_client.js:423:23) 
      at emitNone (events.js:111:20) 
      at TLSSocket.emit (events.js:208:7) 
      at endReadableNT (_stream_readable.js:1056:12) 
      at _combinedTickCallback (internal/process/next_tick.js:138:11) 
      at process._tickCallback (internal/process/next_tick.js:180:9) 
    code: 'ECONNRESET' 
  } 
}
cypress:server:request retrying request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://localhost:8080/en/project/bo/user/register', 
  requestId: 'request9', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 100, 200, 200 ], 
  delay: 0, 
  attempt: 1 
}

[flotwig]

Thanks. Can you share the part of the Cypress test where you upload the avatar, too?

[alegout]

Reading my code, the bug certainly come from this 🤔 (i have a file preview in js):

cy.get('input[name="user_account[avatar]"]')
  .setFile('images/fixture.jpeg');

Cypress.Commands.add(
    'setFile',
    {prevSubject: 'element'},
    function (element, filePath, title, description, copyright) {
        const mimeTypes = {
            jpeg: "image/jpeg",
            jpg: "image/jpeg",
            png: "image/png",
            pdf: "application/pdf",
        };
        const filePathSplitted = filePath.split('.').pop();
        const mimeType = mimeTypes[filePathSplitted] !== undefined
            ? mimeTypes[filePathSplitted]
            : null;
        cy.fixture(filePath, 'base64').then(function(image) {
            Cypress.Blob.base64StringToBlob(image).then(blob => {
                const elementNode = element[0];
                const file = new File([blob], filePath, {type: mimeType});
                const dataTransfer = new DataTransfer();
                dataTransfer.items.add(file);
                elementNode.files = dataTransfer.files;
                elementNode.dispatchEvent(new Event('change', {bubbles: true}));
            });
        });
    }
);

[flotwig]

Nice, I've gotten it to reproduce: 4d5cdbf

Interestingly, your code works fine in 3.3.0 for all files below a certain size - I added some tests for small file uploads and they worked fine, but once I tried uploading this 16MB image through Cypress, it got stuck around 72% every time and eventually times out.

What's the filesize of your failing images, out of curiosity?

[alegout]

Interesting! Thanks

The file: https://ibb.co/ydWyChK

1785x117 and 496 525 octets (500 Ko) on disk

[jennifer-shehane]

I have another person I've had a discussion with that is getting a similar error when upgrading to 3.3.0. Original url has been obfuscated.

Currently waiting for a response on whether they also are using multipart/form-data

cypress:server:request received an error making http request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://sub.domain.company.dev:8000/channel/bind?token=8abc', 
  requestId: 'request396', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 200, 200 ], 
  err: { 
    Error: socket hang up 
    at createHangUpError (_http_client.js:331:15) 
    at TLSSocket.socketOnEnd (_http_client.js:423:23) 
    at emitNone (events.js:111:20) 
    at TLSSocket.emit (events.js:208:7) 
    at endReadableNT (_stream_readable.js:1056:12) 
    at _combinedTickCallback (internal/process/next_tick.js:138:11) 
    at process._tickCallback (internal/process/next_tick.js:180:9) 
    code: 'ECONNRESET' 
  } 
} +4s

[tzimmermann]

I also have the problem that tests that upload files via XHR form request stopped working on 3.3.0, while they were fine on 3.2.0.
The tests are running against an external HTTPS URL, not localhost in my case.
The server responds with a 400 error, which suggests to me that the cypress proxy is not forwarding the file data properly.
I have no custom proxy configuration set up.

It happens both, on Electron 61 and Chrome 74.

I'm using the custom form_request command from here.
The file I'm uploading is 52 KB small.

This is the last debug output I get:

cypress:server:socket backend:request { eventName: 'get:fixture', args: [ 'smallfile.csv', { encoding: 'binary' } ] } +511ms
  cypress:server:timers child received timer id 81 +19ms
  cypress:server:fixture fixture exact name exists /Users/tz/cypress/fixtures/smallfile.csv +0ms
  cypress:server:server Getting remote state: { auth: undefined, props: { port: '443', tld: 'com', domain: 'example' }, origin: 'https://staging.example.com', strategy: 'http', visiting: undefined, domainName: 'example.com', fileServer: null } +4s
  cypress:server:proxy handling proxied request { url: '/g/api/pi/upload/', proxiedUrl: 'https://staging.example.com/g/api/pi/upload/', headers: { host: 'staging.example.com', connection: 'keep-alive', 'content-length': '52553', origin: 'https://staging.example.com', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36', 'content-type': 'multipart/form-data; boundary=----WebKitFormBoundaryY0PpiOqRMKpV0SNe', accept: '*/*', referer: 'https://staging.example.com/__cypress/iframes/integration/testing/access_rights.js', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8', cookie: 'redacted' }, remoteState: { auth: undefined, props: { port: '443', tld: 'com', domain: 'example' }, origin: 'https://staging.example.com', strategy: 'http', visiting: undefined, domainName: 'example.com', fileServer: null } } +3s
  cypress:server:stream_buffer extending buffer to accomodate new chunk { bufferLength: 2048, newBufferLength: 16384 } +0ms
  cypress:server:stream_buffer appending chunk to buffer { bytesWritten: 0, chunkLength: 16384 } +1ms
  cypress:network:agent addRequest called for https://staging.example.com/g/api/pi/upload/ +606ms

[cypress-bot]

The code for this is done in cypress-io/cypress#4241, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[alegout]

thank you @flotwig

[cypress-bot]

Released in 3.3.1.

[sajidit]

I was struggling with the file upload, after updating to version 3.3.1 it's working, Thanks 👍