Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multipart/form-data doesn't work with large files #4240

Closed
flotwig opened this issue May 18, 2019 · 13 comments

Comments

5 participants
@flotwig
Copy link
Member

commented May 18, 2019

Original comment by @alegout: #4235 (comment)

Since Cypress 3.3.0, some users can't upload files via a <form> to a server if the file size is over ~16kB

@flotwig

This comment has been minimized.

Copy link
Member Author

commented May 18, 2019

@alegout re: your last comment: #4235 (comment)

That's failing because of the way the test https server is set up, I added a test for multipart/form-data forms and it works once the https server is set up to accept multipart/form-data. Check out #4241 for the new test.

I will try to reproduce with uploading files.

@alegout

This comment has been minimized.

Copy link

commented May 18, 2019

thanks @flotwig i confirm it's passing.
I will give you my logs.

@alegout

This comment has been minimized.

Copy link

commented May 18, 2019

I got tests for user registration, with an avatar or without.
Test registration without avatar is passing, test with avatar got timeout.

https://gist.github.com/alegout/cb8b92125138964ea58d92e472e8e3f6

cypress:server:request received an error making http request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://localhost:8080/en/project/bo/user/register', 
  requestId: 'request9', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 0, 100, 200, 200 ], 
  err: { 
    Error: socket hang up 
      at createHangUpError (_http_client.js:331:15) 
      at TLSSocket.socketOnEnd (_http_client.js:423:23) 
      at emitNone (events.js:111:20) 
      at TLSSocket.emit (events.js:208:7) 
      at endReadableNT (_stream_readable.js:1056:12) 
      at _combinedTickCallback (internal/process/next_tick.js:138:11) 
      at process._tickCallback (internal/process/next_tick.js:180:9) 
    code: 'ECONNRESET' 
  } 
}
cypress:server:request retrying request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://localhost:8080/en/project/bo/user/register', 
  requestId: 'request9', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 100, 200, 200 ], 
  delay: 0, 
  attempt: 1 
}
@flotwig

This comment has been minimized.

Copy link
Member Author

commented May 18, 2019

Thanks. Can you share the part of the Cypress test where you upload the avatar, too?

@alegout

This comment has been minimized.

Copy link

commented May 19, 2019

Reading my code, the bug certainly come from this 🤔 (i have a file preview in js):

cy.get('input[name="user_account[avatar]"]')
  .setFile('images/fixture.jpeg');
Cypress.Commands.add(
    'setFile',
    {prevSubject: 'element'},
    function (element, filePath, title, description, copyright) {
        const mimeTypes = {
            jpeg: "image/jpeg",
            jpg: "image/jpeg",
            png: "image/png",
            pdf: "application/pdf",
        };
        const filePathSplitted = filePath.split('.').pop();
        const mimeType = mimeTypes[filePathSplitted] !== undefined
            ? mimeTypes[filePathSplitted]
            : null;
        cy.fixture(filePath, 'base64').then(function(image) {
            Cypress.Blob.base64StringToBlob(image).then(blob => {
                const elementNode = element[0];
                const file = new File([blob], filePath, {type: mimeType});
                const dataTransfer = new DataTransfer();
                dataTransfer.items.add(file);
                elementNode.files = dataTransfer.files;
                elementNode.dispatchEvent(new Event('change', {bubbles: true}));
            });
        });
    }
);
@flotwig

This comment has been minimized.

Copy link
Member Author

commented May 20, 2019

Nice, I've gotten it to reproduce: 4d5cdbf

Interestingly, your code works fine in 3.3.0 for all files below a certain size - I added some tests for small file uploads and they worked fine, but once I tried uploading this 16MB image through Cypress, it got stuck around 72% every time and eventually times out.

What's the filesize of your failing images, out of curiosity?

@alegout

This comment has been minimized.

Copy link

commented May 20, 2019

Interesting! Thanks

The file: https://ibb.co/ydWyChK

1785x117 and 496 525 octets (500 Ko) on disk

@flotwig flotwig changed the title multipart/form-data doesn't work over https multipart/form-data doesn't work with large files May 20, 2019

@jennifer-shehane

This comment has been minimized.

Copy link
Member

commented May 21, 2019

I have another person I've had a discussion with that is getting a similar error when upgrading to 3.3.0. Original url has been obfuscated.

Currently waiting for a response on whether they also are using multipart/form-data

cypress:server:request received an error making http request { 
  timeout: null, 
  retryIntervals: [ 0, 100, 200, 200 ], 
  url: 'https://sub.domain.company.dev:8000/channel/bind?token=8abc', 
  requestId: 'request396', 
  retryOnNetworkFailure: true, 
  retryOnStatusCodeFailure: false, 
  delaysRemaining: [ 200, 200 ], 
  err: { 
    Error: socket hang up 
    at createHangUpError (_http_client.js:331:15) 
    at TLSSocket.socketOnEnd (_http_client.js:423:23) 
    at emitNone (events.js:111:20) 
    at TLSSocket.emit (events.js:208:7) 
    at endReadableNT (_stream_readable.js:1056:12) 
    at _combinedTickCallback (internal/process/next_tick.js:138:11) 
    at process._tickCallback (internal/process/next_tick.js:180:9) 
    code: 'ECONNRESET' 
  } 
} +4s
@tzimmermann

This comment has been minimized.

Copy link

commented May 21, 2019

I also have the problem that tests that upload files via XHR form request stopped working on 3.3.0, while they were fine on 3.2.0.
The tests are running against an external HTTPS URL, not localhost in my case.
The server responds with a 400 error, which suggests to me that the cypress proxy is not forwarding the file data properly.
I have no custom proxy configuration set up.

It happens both, on Electron 61 and Chrome 74.

I'm using the custom form_request command from here.
The file I'm uploading is 52 KB small.

This is the last debug output I get:

cypress:server:socket backend:request { eventName: 'get:fixture', args: [ 'smallfile.csv', { encoding: 'binary' } ] } +511ms
  cypress:server:timers child received timer id 81 +19ms
  cypress:server:fixture fixture exact name exists /Users/tz/cypress/fixtures/smallfile.csv +0ms
  cypress:server:server Getting remote state: { auth: undefined, props: { port: '443', tld: 'com', domain: 'example' }, origin: 'https://staging.example.com', strategy: 'http', visiting: undefined, domainName: 'example.com', fileServer: null } +4s
  cypress:server:proxy handling proxied request { url: '/g/api/pi/upload/', proxiedUrl: 'https://staging.example.com/g/api/pi/upload/', headers: { host: 'staging.example.com', connection: 'keep-alive', 'content-length': '52553', origin: 'https://staging.example.com', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36', 'content-type': 'multipart/form-data; boundary=----WebKitFormBoundaryY0PpiOqRMKpV0SNe', accept: '*/*', referer: 'https://staging.example.com/__cypress/iframes/integration/testing/access_rights.js', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8', cookie: 'redacted' }, remoteState: { auth: undefined, props: { port: '443', tld: 'com', domain: 'example' }, origin: 'https://staging.example.com', strategy: 'http', visiting: undefined, domainName: 'example.com', fileServer: null } } +3s
  cypress:server:stream_buffer extending buffer to accomodate new chunk { bufferLength: 2048, newBufferLength: 16384 } +0ms
  cypress:server:stream_buffer appending chunk to buffer { bytesWritten: 0, chunkLength: 16384 } +1ms
  cypress:network:agent addRequest called for https://staging.example.com/g/api/pi/upload/ +606ms
@cypress-bot

This comment has been minimized.

Copy link

commented May 22, 2019

The code for this is done in cypress-io/cypress#4241, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@alegout

This comment has been minimized.

Copy link

commented May 22, 2019

thank you @flotwig

@cypress-bot

This comment has been minimized.

Copy link

commented May 23, 2019

Released in 3.3.1.

@sajidit

This comment has been minimized.

Copy link

commented May 26, 2019

I was struggling with the file upload, after updating to version 3.3.1 it's working, Thanks 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.