Send big data via sample cause connect timeout

[chunlei-wu]

Hi community,
I change the SendBufferLength to 90,000,000 in sample.c, then run the sample in Windows11, the server and the client run on the same PC such as: quicsample.exe -client -unsecure -target:127.0.0.1.

The error, Shut down by transport 0x8041006, almost happens. I change the DisconnectTimeoutMs to 30000, this error also happen sometimes.

I get the start time when server stream receive the first data frame, and the finish time when server stream receive the FIN, it seem server stream spend about 50+ second to receive the 90M data if the transaction success.

Would you mind to help me what's wrong with it and how to get a quick data transaction?
Many thanks!

Best Regards,
Thunder

[TanmayRajore]

I believe Disconnect timeout changes the delayed ack receiving time. you might want to set your idle time settings on the server and client to 0 which sets the idle connection time to indefinite, and then you can choose to close the connection by sending a stream shutdown event in graceful mode or as required check https://github.com/microsoft/msquic/blob/main/docs/api/QUIC_SETTINGS.md

[chunlei-wu]

Many thanks for TanmayRajore,

I retested, it seem the idleTime is no effect on this issue, when I change the disconnect timer to 60s, it will be OK, and I can't find the timeout issue.

And an issue is when I set the SendBufferLength to 90M, it will spend 50+s to send the data to server, it's very slow, when I set the SendBufferLength to 10M, it will be very fast. I guess when the SendBufferLength is bigger than 16M(or another length), we need a difference method to send the big data. But I don't get it now.

Best Regards,
Thunder

[chunlei-wu]

I change the setting SendBufferingEnable to FALSE， and sending 10M data block 10 times instead of 100M one time, it seem OK.
But I don't know the root cause.

[murphypei]

SendBufferingEnable means copy data to MSQUIC buffer and then send it. ref: Sending buffer

[chunlei-wu]

Hi Murphypei,

Let me explain what I mean:
case1: don't set SendBufferEnable, as default, it should be TRUE, send a data block as 100M, it will spend about 50+s to send to server from client, the server and client run on the same PC, and sometimes the connect will failure for timeout
case2: set SendBufferEnable as FASLE, send 10M data block 10 times in turn, it will spend about 3s, it seem faster than the case1.
I don't know why the case1 run failure and so slow

Many thanks for your relay.

[murphypei]

Can you test like this: set SendBufferEnable as FASLE, send a data block as 100M.
how will it speed ?
The ref I put above means SendBufferEnable is the slower send mode.

[chunlei-wu]

I tested as you described above, set SendBufferEnable as FASLE, send a data block as 100M, I don't change the disconnectTimeOutMs. The connect timeout. So it seem the big data block will be hard to handle for msquic, I don't know why.

[murphypei]

em, sorry, I can't answer your question either, I'm new to msquic too.

[nibanks]

Could you please collect the ETL logs via https://github.com/microsoft/msquic/blob/main/docs/Diagnostics.md#wprexe with Full.Light profile ideally, but if that's too big, Datapath.Light instead?

[nibanks]

I tried this locally, and while it doesn't take as long as yours does, it is slower than I'd expect. I've looked into it a bit and it seems to be a bottleneck in part of the OS networking stack on client (not server) Windows editions (things like Defender slowing it down).

[chunlei-wu]

Hi Nibanks,

Many thanks for your work on my issue, it seem you find the root cause, great!
Would you mind to help me how to check or fix this issue on the Windows with more details?
Thanks.

[nibanks]

One suggestion I have is to use MAX_THROUGHPUT profile type instead of LOW_LATENCY. That made the issue go away for the most part on my setup. As far as "how to check" for the issue, I will try to update our TSG with "self serve" instructions to explain the process of diagnosing these issues.

[chunlei-wu]

Great, I use the MAX_THROUGHPUT profile type, it work better.
From client to server: 110~120MB/s
From server to client: 60+MB/s, it seem the change is not effect on this direction?