A large response body gets cut off when using EngineClient

[kdawgwilk]

Environment

Vapor version: 2.0.0-beta.1
Engine version: 2.0.0-beta.1
macOS version: 10.12.4
Swift version: 3.1
Xcode version: 8.3

When using the default droplet client (EngineClient) to make a request that has a large response body, the body will get cut off in the response.

e.g.

let response = try drop.client.get("https://api.github.com/repos/octokit/octokit.rb/releases")
response.json // nil

Upon further investigation dumping the response dump(response) I get the following:

Response
- HTTP/1.1 200 OK
- Headers:
    Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
    X-GitHub-Media-Type: github.v3; format=json
    X-RateLimit-Reset: 1490911130
    ETag: "a1fc205783d327be17f7eafbf3f96002"
    X-Served-By: eef8b8685a106934dcbb4b7c59fba0bf
    Date: Thu, 30 Mar 2017 21:10:37 GMT
    Content-Type: application/json; charset=utf-8
    Content-Length: 59035
    X-Content-Type-Options: nosniff
    Status: 200 OK
    Link: <https://api.github.com/repositories/417862/releases?page=2>; rel="next", <https://api.github.com/repositories/417862/releases?page=2>; rel="last"
    X-GitHub-Request-Id: D6FD:64DE:73E781:93C62B:58DD744D
    Cache-Control: public, max-age=60, s-maxage=60
    Server: GitHub.com
    Vary: Accept-Encoding
    Content-Security-Policy: default-src 'none'
    X-XSS-Protection: 1; mode=block
    X-RateLimit-Limit: 60
    X-RateLimit-Remaining: 58
    Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
    X-Frame-Options: deny
    Access-Control-Allow-Origin: *
- Body:
    [{"url":"https://api.github.com/repos/octokit/octokit.rb/releases/4549630","assets_url":"https://api.github.com/repos/octokit/octokit.rb/releases/4549630/assets","upload_url":"https://uploads.github.com/repos/octokit/octokit.rb/releases/4549630/assets{?name,label}","html_url":"https://github.com/octokit/octokit.rb/releases/tag/v4.6.0","id":4549630,"tag_name":"v4.6.0","target_commitish":"master","name":"","draft":false,"author":{"login":"joeyw","id":478247,"avatar_url":"https://avatars1.githubusercontent.com/u/478247?v=3","gravatar_id":"","url":"https://api.github.com/users/joeyw","html_url":"https://github.com/joeyw","followers_url":"https://api.github.com/users/joeyw/followers","following_url":"https://api.github.com/users/joeyw/following{/other_user}","gists_url":"https://api.github.com/users/joeyw/gists{/gist_id}","starred_url":"https://api.github.com/users/joeyw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joeyw/subscriptions","organizations_url":"https://api.github.com/users/joeyw/orgs","repos_url":"https://api.github.com/users/joeyw/repos","events_url":"https://api.github.com/users/joeyw/events{/privacy}","received_events_url":"https://api.github.com/users/joeyw/received_events","type":"User","site_admin":false},"prerelease":false,"created_at":"2016-11-08T16:22:37Z","published_at":"2016-11-09T08:37:49Z","assets":[],"tarball_url":"https://api.github.com/repos/octokit/octokit.rb/tarball/v4.6.0","zipball_url":"https://api.github.com/repos/octokit/octokit.rb/zipball/v4.6.0","body":"### Library Changes\n\n#829 Sawyer dependency loosened to support new release `'sawyer', '>= 0.5.3', '~> 0.8.0'` Tha

Also trying to parse body.bytes into JSON throws JayError.unexpectedEnd

I was able to get things working by using the FoundationClient instead:

extension FoundationClient: ClientProtocol {
    public convenience init(hostname: String, port: Port, _ securityLayer: SecurityLayer) throws {
        self.init(scheme: "https", hostname: hostname, port: port)
    }
}
drop.client = FoundationClient.self

let response = try drop.client.get("https://api.github.com/repos/octokit/octokit.rb/releases")
response.json // not nil YAAAAYYYY!!!

[hhanesand]

I'm having this issue too...

I'd love to help fix this issue, as this is rather critical. Anything I can do to help?

[tanner0101]

Weird. I have a feeling this is an issue in vapor/sockets.

@hhanesand if you want to submit a failing test case that would help out a ton!

We'll definitely get this fixed before 2.0 goes live.