Improper header termination on non-Windows runtimes

[Wintereise]

In HttpWebClient.cs, we're using StringBuilder#AppendLine to build the header string. The problem with this is the fact that it actually appends <data> + Environment.Newline which works fine in Windows (because NewLine is defined as CRLF).

However, when executing under Unix, this leads to us terminating the line(s) with just LF (\n) instead of what RFC2616 dictates in section 2.2.

CR = <US-ASCII CR, carriage return (13)>
LF = <US-ASCII LF, linefeed (10)>
HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all protocol elements except the entity-body

It's a simple fix for the header-set, and I've contributed a pull request to fix this. It's worth noting that this AppendLine dependency needs removing from anywhere else we might be using it as well, for it is distinctly platform specific.

This would actually cause "strict" HTTPds to issue bad request responses (of which, Apache is one post CVE-2016-8743 patching).

Request samples below:

root@daemon-install-test-0:~# curl -x 127.0.0.1:6100 -v -I http://httpd.apache.org/docs/trunk/getting-started.html -I
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 6100 (#0)
> HEAD http://httpd.apache.org/docs/trunk/getting-started.html HTTP/1.1
> Host: httpd.apache.org
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< Date: Wed, 22 Aug 2018 23:31:52 GMT
Date: Wed, 22 Aug 2018 23:31:52 GMT
< Server: Apache/2.4.18 (Ubuntu)
Server: Apache/2.4.18 (Ubuntu)
< Connection: close
Connection: close
< Content-Type: text/html; charset=iso-8859-1
Content-Type: text/html; charset=iso-8859-1

< 
* Closing connection 0

On the Apache side, this would show:

[Wed Aug 22 13:15:54.671522 2018] [core:trace5] [pid 22094:tid 139914708502272] protocol.c(653): [client 172.98.209.64:35477] Request received from client: GET / HTTP/1.1
[Wed Aug 22 13:15:54.929804 2018] [core:debug] [pid 22094:tid 139914708502272] protocol.c(965): (22)Invalid argument: [client 172.98.209.64:35477] Failed to read request header line Host: 45.77.247.135\n
[Wed Aug 22 13:15:54.929989 2018] [core:debug] [pid 22094:tid 139914708502272] protocol.c(1318): [client 172.98.209.64:35477] AH00567: request failed: error reading the headers
[Wed Aug 22 13:15:54.930150 2018] [http:trace3] [pid 22094:tid 139914708502272] http_filters.c(1128): [client 172.98.209.64:35477] Response sent with status 400, headers:

[Wintereise]

I've raised a pull request for this issue (which fixes it): #484

[Wintereise]

cc: @honfika @justcoding121

[justcoding121]

Thanks. We were aware of this in past from #151 . However I think my changes to use string builder as a cache instead of writing lines to stream directly to improve performance caused this issue.