Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client Writer cw-out refactor #12898

Closed
wants to merge 7 commits into from
Closed

Conversation

icing
Copy link
Contributor

@icing icing commented Feb 8, 2024

Refactoring of the client writer that passes the data to the client/application's callback functions.

  • split out into own source cw-out.[ch] from sendf.c
  • move tempwrite and tempcount from data->state into the context of the client writer
  • redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On paused transfers, this allows to "record" interleaved HEADER/BODY chunks to be "played back" in the same order on unpausing.
  • keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. On exceeding that, return CURLE_TOO_LARGE instead of CURLE_OUT_OF_MEMORY as before.
  • add method to be called when a transfer is DONE to allow writing of any data still buffered
  • when paused, record HEADER writes exactly as they come for later playback. HEADERs are documented to be written one-by-one.

body write buffering

The implementation is capable of buffering body output and aggregate date into larger writes. This is disabled since applications expect to see data at the time it arrives. Should the need from apps to
have curl aggregate date first, we'd need to add a CURLOPT_ for this.

@jay
Copy link
Member

jay commented Feb 9, 2024

All data should be passed to the client immediately if the transfer is not paused. There are clients that process HTTP responses line by line (usually sent by chunk) and stay connected for hours. Even in other cases I'd never want to delay writing to the client.

@icing
Copy link
Contributor Author

icing commented Feb 9, 2024

@jay thanks for your input. That was my suspicion. I will change this PR, not doing this by default and think about adding an option for applications that prefer larger writes.

@icing icing changed the title Client Writer cw-out refactor, buffering option Client Writer cw-out refactor Feb 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants