ERROR (unhandled task failure): read: connection reset by peer (ECONNRESET)

[pevnak]

Hello, in our application we store data (terabytes) fragmented into 100mb files in S3 storage. We have several processes downloading the data and feeding them to the master-process, where they are finally processed (used for training a neural network). The problem we face that we randomly experience following crash, which we cannot handle and recover from it. Is there a principle way, how to deal with it? Can we at least catch the error as an exception?

Thanks for any help. I attach snippet of the error.

ERROR (unhandled task failure): read: connection reset by peer (ECONNRESET)
Stacktrace:
 [1] try_yieldto(::Base.##296#297{Task}, ::Task) at ./event.jl:189
 [2] wait() at ./event.jl:234
 [3] wait(::Condition) at ./event.jl:27
 [4] wait_readnb(::TCPSocket, ::Int64) at ./stream.jl:296
 [5] eof(::TCPSocket) at ./stream.jl:56
 [6] macro expansion at /home/ubuntu/.julia/v0.6/Requests/src/streaming.jl:67 [inlined]
 [7] (::Requests.##6#8{Requests.ResponseStream{TCPSocket},Requests.ResponseParser,Channel{Symbol}})() at ./event.jl:73

[quinnj]

@pevnak, not sure what exactly is going on in Requests.jl, but you could try the HTTP.jl, which aims to be a modernized http package for Julia. In particular, it recently got a thorough overhaul of all the error-handling code specifically to avoid the situation above where "unhandled task failures" happen. The interface is pretty much the same, e.g. Requests.get(url) vs. HTTP.get(url). Happy to answer any questions and help port the code over if that interests you.

[pevnak]

Thank you very much for the answer.
I would need to update this in AWSS3, since I do not call this directly.
Best wishes,
Tomas

[pevnak]

I have found that the version of AWSS3 on master has replaced Requests for HTTP.jl, as you have suggested. Nevertheless I am experiencing some issues with it. Closing for now.

[samoconnor]

Hi @pevnak, I anticipate that AWSCore/AWSS3 will move to HTTP.jl in due course. The interface to HTTP.jl has changed in a number of ways since I did the port that you found on the master branch, so as you have found, it is not currently functional. I am still using my local copy of a slightly older version of HTTP.jl try to give it a work out when I can (e.g. JuliaWeb/HTTP.jl#98). However, it might be a few weeks before I have the time to update AWSCore to work with the latest HTTP.jl.

@quinnj, Do you expect the interface to change much again after v0.5.0 ?

[quinnj]

No, I don't expect much change in the near future.

[samoconnor]

Thx @quinnj, could you please tag and update METADATA.jl for: JuliaWeb/HTTP.jl@v0.5.0...master ?

I'd like to make another attempt at using HTTP.jl in AWSCore.jl.

[quinnj]

@samoconnor, just finished tagging a new release.