Latest master failing

[pevnak]

Hello,

I have checked out the latest master of AWSCore and AWSS3, because of the issue with Requests
JuliaWeb/Requests.jl#171
Unfortunately, even the simplest listing the content of the bucket as

using AWSCore
using AWSS3
aws = AWSCore.aws_config()
AWSS3.s3_list_objects("bucket_name")

failst with

ERROR: MethodError: no method matching escape(::Array{Pair,1})
Closest candidates are:
  escape(::Any, ::Any, ::Array{String,1}) at /.julia/v0.6/HTTP/src/uri.jl:173
  escape(::Any, ::Any, ::Any) at /.julia/v0.6/HTTP/src/uri.jl:171
  escape(::Array{UInt8,1}) at /.julia/v0.6/HTTP/src/uri.jl:169
  ...
Stacktrace:
 [1] macro expansion at /.julia/v0.6/AWSS3/src/AWSS3.jl:484 [inlined]
 [2] macro expansion at /.julia/v0.6/Retry/src/repeat_try.jl:198 [inlined]
 [3] s3_list_objects(::Dict{Symbol,Any}, ::String, ::String) at /.julia/v0.6/AWSS3/src/AWSS3.jl:468
 [4] s3_list_objects(::Dict{Symbol,Any}, ::String) at /.julia/v0.6/AWSS3/src/AWSS3.jl:453
 [5] s3_list_objects(::String) at/.julia/v0.6/AWSS3/src/AWSS3.jl:491

when I have checked out the latest master of HTTP, I got this error

ERROR: MethodError: no method matching request(::HTTP.Client, ::HTTP.Request, ::HTTP.RequestOptions; stream=false, verbose=false)
Closest candidates are:
  request(::HTTP.Client, ::HTTP.Request, ::HTTP.RequestOptions, ::Bool, ::Array{HTTP.Response,1}, ::Int64, ::Bool) at /.julia/v0.6/HTTP/src/client.jl:299 got unsupported keyword arguments "stream", "verbose"
  request(::HTTP.Client, ::HTTP.Request; opts, stream, history, retry, verbose, args...) at /.julia/v0.6/HTTP/src/client.jl:341
  request(::HTTP.Client, ::Any, ::HTTP.URIs.URI; headers, body, stream, verbose, args...) at /.julia/v0.6/HTTP/src/client.jl:357
  ...
Stacktrace:
 [1] macro expansion at /.julia/v0.6/AWSS3/src/AWSS3.jl:484 [inlined]
 [2] macro expansion at /.julia/v0.6/Retry/src/repeat_try.jl:198 [inlined]
 [3] s3_list_objects(::Dict{Symbol,Any}, ::String, ::String) at /.julia/v0.6/AWSS3/src/AWSS3.jl:468

It seems to me that the versioning is little bit messed up. Would it be possible to tag new stable versions?

Thanks a lot,
Tomas

[pevnak]

I think that the problem is somewhere in function http_attempt(request::AWSRequest) in AWSCore/src/http.jl,

where it does not match the API of HTTP.jl. I have tried to fix it to

function http_attempt(request::AWSRequest)
    response = HTTP.request(HTTP.Client(STDOUT),
                            HTTP.Request(request[:verb],
                                         Int16(1), Int16(1),
                                         HTTP.URI(request[:url]),
                                         request[:headers],
                                         FIFOBuffer(request[:content])),
                            HTTP.RequestOptions(connecttimeout = 600,
                                                readtimeout = 600,
                                                allowredirects = false,
                                                retries = 0),
                            get(request, :return_stream, false),
                            HTTP.Response[],
                            0,
                            debug_level > 1)

    if !http_ok(response)
        throw(HTTPException(response))
    end

    return response
end

But I have received error

ERROR: HTTP.StatusError: received a '400 - Bad Request' status in response
Stacktrace:
 [1] macro expansion at /.julia/v0.6/AWSS3/src/AWSS3.jl:484 [inlined]
 [2] macro expansion at /.julia/v0.6/Retry/src/repeat_try.jl:198 [inlined]
 [3] s3_list_objects(::Dict{Symbol,Any}, ::String, ::String) at /.julia/v0.6/AWSS3/src/AWSS3.jl:468

from which I guess I did something wrong and my experience ends here.
Thanks for help.

[pevnak]

So the last error was caused by the fact that my S3 storage was in different area than the default. It took me quite while to track that problem down. It would be therefore nice to improve the error messages, but I do not know how.

I have also encountered one more discrepancy between API of AWSCore and HTTP.jl. I have created a quick and dirty fix. I would make a PR out of it, but just to let other people know, where are the changes. I truly think that the fix is not correct from the point of view of the library.

[samoconnor]

Hi @pevnak, are these issue all resolved now with latest HTTP.jl and AWSCore.jl?

[pevnak]

Yes. I would mark it as resolved. It is running continuously for a day and it seems to work flawlessly.

Thank you very much for the support. I would not be able to do it by myself.
Tomas