Concurrent requests seem to break testWithApplication

[lancelet]

When Warp is sent concurrent requests in testWithApplication, it often fails with a timeout.

Test code to replicate the problem is included below (as a stack script). For me, this test very occasionally succeeds. However, the vast majority of the time, it fails.

#!/usr/bin/env stack
{- stack
  script
  --resolver lts-10.3
  --package async
  --package bytestring
  --package http-client
  --package http-types
  --package wai
  --package warp
-}
{-# LANGUAGE OverloadedStrings #-}

import           Control.Concurrent         (threadDelay)
import           Control.Concurrent.Async   (wait, withAsync)
import qualified Data.ByteString.Lazy       as LBS
import qualified Data.ByteString.Lazy.Char8 as LC8
import           Data.Monoid                ((<>))
import           Network.HTTP.Client        (defaultManagerSettings, httpLbs,
                                             newManager, parseRequest, port,
                                             responseBody)
import           Network.HTTP.Types         (status200)
import           Network.Wai                (Application, responseLBS)
import           Network.Wai.Handler.Warp   (Port, testWithApplication)



main :: IO ()
main = testWithApplication (pure app) testAction



testAction :: Port -> IO ()
testAction p = do

    manager <- newManager defaultManagerSettings
    requestWithoutPort <- parseRequest "http://localhost/"
    let request = requestWithoutPort { port = p }

    withAsync (httpLbs request manager) $ \a1 -> do
        -- uncomment the line below to observe correct responses
        -- threadDelay 100000
        withAsync (httpLbs request manager) $ \a2 -> do

            r1 <- responseBody <$> wait a1
            r2 <- responseBody <$> wait a2

            LC8.putStrLn $ "r1 = " <> r1
            LC8.putStrLn $ "r2 = " <> r2



app :: Application
app _request respond = respond $ responseLBS status200 [] "Response"

Example of the observed failure:

$ ./warp-problem.hs                   
warp-problem.hs: HttpExceptionRequest Request {
  host                 = "localhost"
  port                 = 50925
  secure               = False
  requestHeaders       = []
  path                 = "/"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 NoResponseDataReceived

Expected output:

$ ./warp-problem.hs
r1 = Response
r2 = Response

Relevant system information (some is not relevant because the above stack script is self-contained):

ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G18013
/Users/<redacted>/.nix-profile/bin/uname
Darwin C02MC13TFD58 15.6.0 Darwin Kernel Version 15.6.0: Mon Nov 13 21:58:35 PST 2017; root:xnu-3248.72.11~1/RELEASE_X86_64 x86_64 i386 MacBookPro11,2 Darwin
/Users/<redacted>/.local/bin/stack
Version 1.6.1, Git revision f25811329bbc40b0c21053a8160c56f923e1201b (5435 commits) x86_64 hpack-0.20.0

[snoyberg]

I just pushed a commit to master to ignore some exception in testWithApplication, see e8aa56e. Can you confirm that this addresses the behavior you saw?

[lancelet]

Unfortunately I'm still seeing the same thing.

I've created a repo which contains the test code and points to the latest commit:
https://github.com/lancelet/warp-problem
(Using a git submodule.)

Is this perhaps macOS-specific, or are you also able to replicate the problem with the above repo?

[snoyberg]

OK, pretty sure I got it with c548f55. Turns out that the backlog was being set to 1. Can you try out master now?

[lancelet]

@snoyberg thanks for the fix. I can confirm that I'm able to run my example repeatedly without problems. 😄

[snoyberg]

Cool, new version released to Hackage.

…

On Thu, Feb 15, 2018 at 2:55 AM, Jonathan Merritt ***@***.***> wrote: @snoyberg <https://github.com/snoyberg> thanks for the fix. I can confirm that I'm able to run my example repeatedly without problems. 😄 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#671 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADBB4jlCYOC-1n4LjKnpeXCQV2N9Xt3ks5tU4BmgaJpZM4RzOQQ> .