HTTP: implement trailing headers for chunked transfers

[aneutron]

This patch adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION options that allow a callback based approach to sending trailing headers with chunked transfers.

Following the everything is a state machine mantra, it's four stages state machine that traps right before sending the final CRLF, calls the trailer function that fills a curl_slist with the trailers, compiles them into a single buffer, and proceeds to read from the buffer as if reading from the read callback, then jumps back into the next main state as if nothing happened.

The test server (sws) was updated to take into account the detection of the end of transfer in the case of trailing headers presence.

Test 1591 checks that trailing headers can be sent using libcurl.

(I'm very sorry I had to remake this PR, the first one was made to merge my master branch into the curl master branch, it was a rookie mistake that made rebasing difficult ... That and the lack of tests, half-baked docs and tests that didn't pass. Suffice to say that mistakes were made. I sincerely apologize. All the remarks on the previous PR were addressed.)

[bagder]

Nice work! I've mostly found minor nits and details to remark on.

[bagder]

@danielgustafsson it seems the travis check for copyright year mismatch doesn't work on new files?

[bagder]

"custom pointer passed to the trailing headers callback" maybe? We want it to be short and sweet. It also makes sense to have all *DATA options described using similar wording.

[bagder]

A little bikeshedding here perhaps, but...

I'm not convinced it is a good idea to include HTTP in the name. What if there's another protocol with a similar concept that we could use this callback for at a later point in time? Maybe just CURLOPT_TRAILERFUNCTION ? Most of the existing *FUNCTION names are already named without a separating underscore.

[bagder]

Since we discourage // comments everywhere (for C89 portability), we probably should use them in the man page example either...

[bagder]

Here too I think this should be as short as possible that still conveys what it does:

"set callback for sending trailing headers" perhaps?

[bagder]

When splitting long expressions, we usually prefer to split the lines with the operator on the previous line, and I would recommend parentheses:

(data->req.upload_chunky &&
 (data->state.trailers_state == HTTP_TRAILERS_NONE))

[bagder]

this looks funnily indented

[bagder]

not two-space indentation

[bagder]

no point in explaining what it isn't...

[bagder]

maybe add spaces to make this more readable?

return trailers_buf->size_used - data->state.trailers_bytes_sent;

[aneutron]

I think I addressed all the remarks:

• Replaced CULROPT_HTTP_TRAILER* with CURLOPT_TRAILER*
• Renamed the trailer state enum members from HTTP_TRAILER_* to TRAILER_*
• Added two defs (CURL_TRAILERFUNC_OK & CURL_TRAILERFUNC_ABORT) to be used as return codes for the trailer function that now returns an integer.
• Fixed the dates and releases on the docs, and edited the docs as suggested.
• Fixed the weird indentations that slipped.
• (I'm sorry I just noticed this a day after I addressed everything else) Fixed the commit message to reflect these changes.

[mkauf]

Please fix the typo in the commit message: "chuncked" --> "chunked".

[aneutron]

@mkauf Thanks for spotting that !
@bagder Should I leave HTTP: in the commit message ?

[bagder]

Should I leave HTTP: in the commit message?

Yes I think so, since this is after all only for HTTP (now).

[bagder]

Thanks!

[aneutron]

Thanks a lot ! 😄