Proxy buffers Transfer-Encoding: chunked responses and breaks streams

[nLight]

Hi friends!

We have a XmlHttpRequest that never finishes. We're using it as a stream parsing responseText on every onProgress event.

The problem is that we never get the last chunk as it would be sitting somewhere buffered. We can't really control the server behaviour here and I wonder if the buffer can be flushed after each chunk?

• Operating System: Linux
• Cypress Version: 3.1.3
• Browser Version: all of them

Is this a Feature or Bug?

bug

Current behavior:

cypress waits a request to complete

Desired behavior:

Cypress should propagate onProgress events and update responseText accordingly

How to reproduce:

• an endpoint with Transfer-Encoding: chunked that sends messages one after another and doesn't close throughout the session.
• browser connection that reads those messages listening to onProgress

[brian-mann]

I'm sure this is a bug and it'll be fixed by #687

[tdaniely]

Any ideas on how to workaround this currently? I have a big init call on most page loads that's chunked.
I'm trying to stub the route with the result of a cy.request(..), but I'm not getting there.
If I put the response in a fixture, then it goes through, but a static isn't enough in this situation.

Edit: Eventually opted to just change the app itself to disable chunked encoding during tests.

[nLight]

There are quite some issues that have one common thing among them: Transfer-Encoding: chunked. In our case we have a persistent connection that never closes. Looks like there's a some kind of buffer. We never get the last chunk of data. I assume there's a proxy that waits for another chunk or the connection to be closed by the server to flush the buffer.

[nLight]

@jennifer-shehane I wouldn't call it a feature though.

[nLight]

I updated description as I got some insights.

[parostatkiem-zz]

I'm not sure if this is exactly the same problem but I also encountered some issues with a chunked HTTP request which should stay open forever. It's used for getting logs from a Kubernetes cluster - not our decision to use such a weird solution...

Everything works well in all browsers outside Cypress but the request clearly doesn't get/trigger any updates in Cypress (tested Chrome, Chrome headless and Electron), even after a very long time (30s).
I can even re-enter the same url inside Cypress (by just clicking it) and there are still no logs. However, once the URL is visited in a browser outside Cypress, the logs show immediately. There are no logs or errors in the dev console.

Any idea how to debug it further or... solve the problem?

This is the code responsible for handling the request in case You need more technical details.

[Sawthis]

Any ideas how could we fix this bug?

[parostatkiem-zz]

@jennifer-shehane could you have a look, please?

[yupitomets]

Any updates on this issue?

[xvotro02]

I have same problem as mentioned before. Any updates?

Our HTTP request is chunked, response type text/event stream. We use Angular frontend where I tried native EventStream, fetch and nx sse client implementation. With EventStream I correctly see data in chrome dev tools. But nothing works correctly in Cypress.

Without adding to cypress everything works fine - I see immediately every chunk (Server Sent Event) on frontend. But when I add the URL to the Cypress Proxy, e.g. by adding to intercept command, responses are buffered and are waiting untill the connection ended, e.g. when connection timeouts on backend. Then all data come in one big chunk.

At the begining we had same problem on nginx proxy where we needed to disable buffering for this kind of requests.

[wortkrieg]

+1. This is causing me massive headache. Is there any workaround?

I can also see that there might be some connection to Content-Encoding: gzip when comparing to a request in normal chrome browser:
Chrome (non-cypress) says nothing in the response header about Content-Encoding and Transfer-Encoding.

HTTP/1.1 200 OK X-Content-Type-Options: nosniff content-length: 11084 content-type: text/javascript cache-control: no-store connection: close

Cypress Chrome (and firefox) don't work and have Accept-Encoding, Content-Encoding gzip and Transfer-Encoding chunked in the response. It seems like cypress requests it or internally forwards the data:

HTTP/1.1 200 OK x-content-type-options: nosniff content-type: text/javascript cache-control: must-revalidate, public, max-age=5184000 Vary: Accept-Encoding Content-Encoding: gzip Date: Thu, 25 May 2023 08:23:29 GMT Connection: keep-alive Keep-Alive: timeout=5 Transfer-Encoding: chunked

Any way to change this behaviour on the cypress side?

[nagash77]

@wortkrieg or anyone else on this thread, are you able to provide a reproducible example using Cypress Test Tiny that would help our engineers investigate this issue?

[nagash77]

I am going to close this issue as it is quite old at this point. If folks are seeing this behavior in current versions of Cypress please open a new ticket with a reproducible example and we will be happy to investigate.