New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 Example broken #466

Open
reactormonk opened this Issue May 28, 2018 · 20 comments

Comments

Projects
None yet
@reactormonk

reactormonk commented May 28, 2018

This line now breaks with a ConnectionClosed. It worked before with lts-9.X, doesn't anymore with nightly stackage.

From what I know, it's an issue with the connection being closed too fast in this part of the code. Instead, the MonadResource instance from outside should be used.
https://github.com/brendanhay/amazonka/blob/develop/examples/src/Example/S3.hs#L76

@AusCoder

This comment has been minimized.

AusCoder commented Jun 2, 2018

I ran into the same problem using amazonka-1.6.0. Yes, it would appear the connection is being closed too soon. Here is my best guess as to why: in Network.AWS.Internal.HTTP, the http call is made with

perform Env{..} x = liftIO $ catches (runResourceT go) handlers

My guess is that the runResourceT is closing the connection before we can actually use the conduit.

However, I think the problem has been fixed. This line was reverted recently, see here. It currently works with a local copy of the package from master.

@ababkin

This comment has been minimized.

ababkin commented Jun 4, 2018

can also confirm this is broken on 06/02 nightly stackage (1.6.0) and that it is fixed at the tip of develop

@blender

This comment has been minimized.

blender commented Jun 19, 2018

Adding my voice to the others, broken on lts 11.13 .

@utdemir

This comment has been minimized.

Contributor

utdemir commented Jun 24, 2018

@brendanhay Is possible to have a new release, since this is already fixed on develop?

@naushadh

This comment has been minimized.

Contributor

naushadh commented Jul 7, 2018

For the future, something like #477 can help prevent such regressions.

@jdreaver

This comment has been minimized.

jdreaver commented Jul 11, 2018

FYI it appears this is the PR that fixed the line @AusCoder is talking about #464

@dfithian

This comment has been minimized.

dfithian commented Jul 12, 2018

Would also LOVE a release

@ryota-ka

This comment has been minimized.

ryota-ka commented Jul 30, 2018

Are there any workarounds this problem until the new version is released?
(Or I just have to wait for it?)

@blender

This comment has been minimized.

blender commented Jul 30, 2018

@ryota-ka if you use stack you can use the latest master like so:
https://github.com/blender/Rome/blob/master/stack.yaml#L38-L47

@ryota-ka

This comment has been minimized.

ryota-ka commented Jul 30, 2018

@blender I've just tried it and it works, thanks!

btw I hope the new version will be released in the near future 🙏

@newhoggy

This comment has been minimized.

newhoggy commented Jul 30, 2018

@brendanhay can we have a new release please. 🙏

@dsturnbull

This comment has been minimized.

dsturnbull commented Jul 30, 2018

Would really love a patch release 🙏

@MichaelXavier

This comment has been minimized.

Collaborator

MichaelXavier commented Aug 2, 2018

Relevant to other folks in this thread, see #484 for a status update. @brendanhay is going to be committing some time to work on this project in late August/early September. If anyone is a heavy user that wants to help with the maintenance burden of the project, release management, etc, perhaps get in touch.

@greghill

This comment has been minimized.

greghill commented Aug 23, 2018

EDIT: disregard me

Is reading from S3 completely broken then? I am not able to reproduce these workarounds even at the tip of develop

*** Exception: HttpExceptionRequest Request {
  host                 = "s3.amazonaws.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("Host","s3.amazonaws.com"),("X-Amz-Date","20180823T211638Z"),("X-Amz-Content-SHA256","e3b0c44298fc1c149a41e4649b934ca495991b7852b855"),("Authorization","<REDACTED>")]
  path                 = "/greg"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 0
  responseTimeout      = ResponseTimeoutMicro 70000000
  requestVersion       = HTTP/1.1
}
 ConnectionClosed

using stack:

extra-deps:
- git: git@github.com:brendanhay/amazonka
  commit: 248f7b2a7248222cc21cef6194cd1872ba99ac5d
  subdirs:
    - amazonka
    - core
    - amazonka-s3
resolver: lts-12.7
@blender

This comment has been minimized.

blender commented Aug 23, 2018

@greghill did you try master with the configuration I posted? works for me.

@greghill

This comment has been minimized.

greghill commented Aug 24, 2018

@blender it turns out I was trying to read the conduit after already finishing runResourceT so I got the same error as this one even after applying your fix. I fixed that and now your configuration works.

The joys of software

result <- runResourceT . runAWST env . within NorthVirginia . send $ getCmd
resultText <- (result ^. gorsBody) `sinkBody` sinkLbs

This is a subtle bug. In the future it would be beautiful if this could be prevented at the type level in this library.

@rblnguyen

This comment has been minimized.

rblnguyen commented Sep 8, 2018

thank you @blender. The commit you pointed out work great.
git: git@github.com:brendanhay/amazonka.git
commit: c9d8a62
subdirs:

  • amazonka
  • core
  • amazonka-s3
@neongreen

This comment has been minimized.

neongreen commented Sep 19, 2018

A warning: I'm seeing significant slowdowns when talking to SQS, introduced by this commit: 2688190. I'm investigating currently; maybe it would be prudent to avoid using the HEAD commit for now.

@neongreen

This comment has been minimized.

neongreen commented Sep 19, 2018

I filed an issue: #490. Looks like 1.5.0 is the only version of amazonka that isn't broken one way or the other.

@ianbamforth

This comment has been minimized.

ianbamforth commented Oct 24, 2018

I also had the issue where the HttpExceptionRequest was returned as above. Using the commit as advised by @blender got me past that (thanks!), but the file written to disk is always empty.

downloadAndReadInFile awsEnv loc@(S3Location bucket key) = AWS.runResourceT $ do
  downloadResult <- try $ AWS.runAWS awsEnv $ do
    res <- AWS.send (S3.getObject (S3.BucketName bucket) (S3.ObjectKey key))
    AWS.sinkBody (res ^. S3.gorsBody) (sinkSystemTempFile fileName)
  case downloadResult of
    Left ex -> pure $ Left ex
    Right fp -> liftIO $ mapLeft SomeException <$> readPlainText fp ext
  where
  ext = takeExtension fileName
  fileName = unpack $ s3FileName loc

Any idea why this could be happening?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment