Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrent requests seem to break testWithApplication #671

Closed
lancelet opened this Issue Jan 31, 2018 · 5 comments

Comments

Projects
None yet
2 participants
@lancelet
Copy link

lancelet commented Jan 31, 2018

When Warp is sent concurrent requests in testWithApplication, it often fails with a timeout.

Test code to replicate the problem is included below (as a stack script). For me, this test very occasionally succeeds. However, the vast majority of the time, it fails.

#!/usr/bin/env stack
{- stack
  script
  --resolver lts-10.3
  --package async
  --package bytestring
  --package http-client
  --package http-types
  --package wai
  --package warp
-}
{-# LANGUAGE OverloadedStrings #-}

import           Control.Concurrent         (threadDelay)
import           Control.Concurrent.Async   (wait, withAsync)
import qualified Data.ByteString.Lazy       as LBS
import qualified Data.ByteString.Lazy.Char8 as LC8
import           Data.Monoid                ((<>))
import           Network.HTTP.Client        (defaultManagerSettings, httpLbs,
                                             newManager, parseRequest, port,
                                             responseBody)
import           Network.HTTP.Types         (status200)
import           Network.Wai                (Application, responseLBS)
import           Network.Wai.Handler.Warp   (Port, testWithApplication)



main :: IO ()
main = testWithApplication (pure app) testAction



testAction :: Port -> IO ()
testAction p = do

    manager <- newManager defaultManagerSettings
    requestWithoutPort <- parseRequest "http://localhost/"
    let request = requestWithoutPort { port = p }

    withAsync (httpLbs request manager) $ \a1 -> do
        -- uncomment the line below to observe correct responses
        -- threadDelay 100000
        withAsync (httpLbs request manager) $ \a2 -> do

            r1 <- responseBody <$> wait a1
            r2 <- responseBody <$> wait a2

            LC8.putStrLn $ "r1 = " <> r1
            LC8.putStrLn $ "r2 = " <> r2



app :: Application
app _request respond = respond $ responseLBS status200 [] "Response"

Example of the observed failure:

$ ./warp-problem.hs                   
warp-problem.hs: HttpExceptionRequest Request {
  host                 = "localhost"
  port                 = 50925
  secure               = False
  requestHeaders       = []
  path                 = "/"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 NoResponseDataReceived

Expected output:

$ ./warp-problem.hs
r1 = Response
r2 = Response

Relevant system information (some is not relevant because the above stack script is self-contained):

ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G18013
/Users/<redacted>/.nix-profile/bin/uname
Darwin C02MC13TFD58 15.6.0 Darwin Kernel Version 15.6.0: Mon Nov 13 21:58:35 PST 2017; root:xnu-3248.72.11~1/RELEASE_X86_64 x86_64 i386 MacBookPro11,2 Darwin
/Users/<redacted>/.local/bin/stack
Version 1.6.1, Git revision f25811329bbc40b0c21053a8160c56f923e1201b (5435 commits) x86_64 hpack-0.20.0

snoyberg added a commit that referenced this issue Jan 31, 2018

@snoyberg

This comment has been minimized.

Copy link
Member

snoyberg commented Jan 31, 2018

I just pushed a commit to master to ignore some exception in testWithApplication, see e8aa56e. Can you confirm that this addresses the behavior you saw?

@lancelet

This comment has been minimized.

Copy link
Author

lancelet commented Feb 4, 2018

Unfortunately I'm still seeing the same thing.

I've created a repo which contains the test code and points to the latest commit:
https://github.com/lancelet/warp-problem
(Using a git submodule.)

Is this perhaps macOS-specific, or are you also able to replicate the problem with the above repo?

snoyberg added a commit that referenced this issue Feb 7, 2018

Reuse bindRandomPortTCP #671
Turns out that the real issue was having too small a backlog. But may as well
reuse the existing random port functionality in streaming-commons.
@snoyberg

This comment has been minimized.

Copy link
Member

snoyberg commented Feb 7, 2018

OK, pretty sure I got it with c548f55. Turns out that the backlog was being set to 1. Can you try out master now?

@lancelet

This comment has been minimized.

Copy link
Author

lancelet commented Feb 15, 2018

@snoyberg thanks for the fix. I can confirm that I'm able to run my example repeatedly without problems. 😄

@lancelet lancelet closed this Feb 15, 2018

@snoyberg

This comment has been minimized.

Copy link
Member

snoyberg commented Feb 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.