204 with no body hangs on second attempt after reading body

[andreausu]

Hi,

we've run into a very weird issue using hackney (via HTTPoison), basically we're getting an error whenever we make two requests towards the same IP address in quick succession where the first request return 204 with an empty body.

I setup a public test url where you can reproduce the issue, here's the controller code (it uses phoenix):

def debug_empty_body(conn, _params) do
    conn
    |> put_status(204)
    |> json(nil)
end

Heres the public test url: https://backoffice-staging.prima.it/debug/emptybody

You can reproduce the problem by executing this in quick succession:

iex(46)> url = "https://backoffice-staging.prima.it/debug/emptybody"
iex(47)> {:ok, status_code, headers, client} = :hackney.request(:get, url)
{:ok, 204,
 [{"Date", "Fri, 08 Sep 2017 08:25:34 GMT"},
  {"Content-Type", "application/json; charset=utf-8"}, {"Content-Length", "4"},
  {"Connection", "keep-alive"},
  {"Set-Cookie",
   "__cfduid=d7069bf774c5da118496cc760b8c3963d1504859134; expires=Sat, 08-Sep-18 08:25:34 GMT; path=/; domain=.prima.it; HttpOnly"},
  {"Set-Cookie",
   "AWSALB=mqdXVI7J+5GMDPTBM7ig8XnWcPITaBmVBEhqx8dw2mWTfetmj8UkfYqbf/XdfNUtu3m27xV1tx5zMf5J9Hkl1fY9wqjQ4WKXcunYa/rNAN7e85CGgBwY3gV+4Wck; Expires=Fri, 15 Sep 2017 08:25:34 GMT; Path=/"},
  {"cache-control", "max-age=0, private, must-revalidate"},
  {"x-request-id", "gekfc0gl912cuv56ol6sp4tkko7ubvvp"},
  {"x-frame-options", "SAMEORIGIN"}, {"x-xss-protection", "1; mode=block"},
  {"x-content-type-options", "nosniff"},
  {"Strict-Transport-Security", "max-age=15552000; preload"},
  {"Server", "cloudflare-nginx"}, {"CF-RAY", "39b08497d8d70e36-MXP"}],
 #Reference<0.369302331.3057123329.199753>}
iex(48)> :hackney.body(client)
{:ok, ""}
iex(49)> {:ok, status_code, headers, client} = :hackney.request(:get, url)
** (MatchError) no match of right hand side value: {:error, :closed}

Some interesting facts:

• The issue doesn't manifest itself if I don't try to read the body (:hackney.body(client)).
• I've tried doing the same using several different HTTP clients from different programming languages including curl and I wasn't able to reproduce the issue in any of those.
• If I skip the load balancer (AWS Application Load Balancer) in the middle and go straight to the application using the instance private IP address on a non standard port (eg: http://10.33.25.104:10034/debug/emptybody) the second request works just fine.
• Erlang version: I've tested both 19 and 20
• Hackney version: I've tested both 1.9.0 and 1.8.6

I've captured a tcpdump of the issue (the first request is executed at 10:01:55, the second one at 10:02:00):

Thank you!