Unable To Connect to Endpoint (Curl returned error code 28)

[chrishadiono]

I am trying to upload a .jpg to my bucket and get this error
PutObject error: Unable to connect to endpoint

My function looks like this to upload the image to S3

inline void 
  uploadToS3(Aws::String bucket_name, Aws::String key_name, Aws::String file_path)
  {
    Aws::SDKOptions options;
    Aws::InitAPI(options);
    Aws::Client::ClientConfiguration clientConfig;
    clientConfig.scheme = Aws::Http::Scheme::HTTPS;
    clientConfig.region = Aws::Region::US_WEST_1;
    Aws::S3::S3Client s3_client(clientConfig);
    clientConfig.connectTimeoutMs = 30000;
    clientConfig.requestTimeoutMs = 600000;
    Aws::S3::Model::PutObjectRequest object_request;
    object_request.WithBucket(bucket_name).WithKey(key_name);

    auto input_data = Aws::MakeShared<Aws::FStream>("PutObjectInputStream", 
    file_name.c_str(), std::ios_base::in | std::ios_base::binary | std::ios_base::trunc);

    object_request.SetBody(input_data);
    object_request.SetContentType("image/jpeg");
    object_request.SetContentLength(static_cast<long>(51908));
    auto put_object_outcome = s3_client.PutObject(object_request);

    if (put_object_outcome.IsSuccess())
    {
      LOG_INFO("Uploaded to bucket");
    }
    else
    {
      std::cout << "PutObject error: " <<
      put_object_outcome.GetError().GetExceptionName() << " " <<
      put_object_outcome.GetError().GetMessage() << std::endl;
    }
    Aws::ShutdownAPI(options);
  }

I have to do SetContentLength otherwise I get a
BadRequest Message: An error occurred when parsing the HTTP request. error

I use this function like so:

Aws::String file_path = "testimage1.jpg";
Aws::String bucket_name = "dummy";
Aws::String key_name = "key"; //secret key
uploadToS3(bucket_name, key_name, file_path);

I attach the log where I get the error code 28 error. I'm not sure how to fix this error. thanks!

aws_sdk_2018-06-27-23.log

[marcomagdy]

Seems like a firewall issue. The SDK, by default, uses virtual addressing, meaning, it will connect to the following host:
fetchdummy.s3.us-west-1.amazonaws.com make sure that endpoint is reachable from your machine using curl or similar tools

[chrishadiono]

Thanks for the reply, I just used curl and was able to upload fine so I believe it is not a firewall issue. I still believe it may be related to the date header though i havent come up with a fix yet

[chrishadiono]

Ah, it wouldn't be the time actually I didn't realize it was GMT. The script I used that worked successfully is here: https://gist.github.com/chrismdp/6c6b6c825b07f680e710

S3KEY="my aws key"
S3SECRET="my aws secret" # pass these in

function putS3
{
  path=$1
  file=$2
  aws_path=$3
  bucket='my-aws-bucket'
  date=$(date +"%a, %d %b %Y %T %z")
  acl="x-amz-acl:public-read"
  content_type='application/x-compressed-tar'
  string="PUT\n\n$content_type\n$date\n$acl\n/$bucket$aws_path$file"
  signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)
  curl -X PUT -T "$path/$file" \
    -H "Host: $bucket.s3.amazonaws.com" \
    -H "Date: $date" \
    -H "Content-Type: $content_type" \
    -H "$acl" \
    -H "Authorization: AWS ${S3KEY}:$signature" \
    "https://$bucket.s3.amazonaws.com$aws_path$file"
}

for file in "$path"/*; do
  putS3 "$path" "${file##*/}" "/path/on/s3/to/files/"
done

I am having trouble identifying what information my s3 upload function is missing that this script contains

[marcomagdy]

The script hits the us-east-1 endpoint. Your bucket is in a different region and hits a different endpoint.

[chrishadiono]

Oh that's weird then. The script shouldn't have worked then since my bucket is in the us-west-1 region right? I just reran my uploadToS3 function and set the region to us-east-1 and it gave me an error that it is expecting 'us-west-1' so it is identifying the bucket in the right region, why would it accept the script if it uses us-east-1

[chrishadiono]

I identified that without this line acl="x-amz-acl:public-read", the script does not work. Does that have any relation to my uploadToS3 function not working?

[chrishadiono]

I also was able to use the python aws SDK to upload a .jpg with my default region set as us-west-1, but it would be much more convenient for me if I can get it working with the cpp sdk!

[marcomagdy]

Your logs show that curl returns with a timeout error after 3 seconds. However, your config shows clientConfig.connectTimeoutMs = 30000; which is 30 seconds. Depending on your location from us-west-1, you might need more than 3 seconds. Can you please verify the number you pasted above?

[DEBUG] 2018-06-27 23:33:05 CurlHttpClient [140427369899776] Obtained connection handle 0x7fb7c09c44a0
[ERROR] 2018-06-27 23:33:08 CurlHttpClient [140427369899776] Curl returned error code 28

[chrishadiono]

Yes that number is correct. I attached another log here (I set the maxContainers to 30 in this example just seeing if it would magically fix)
aws_sdk_2018-06-28-23.log

[marcomagdy]

Doh. You set the timeout on the config AFTER you already passed it to s3client.

[chrishadiono]

Ahhhh woops good catch! It does timeout after 30 seconds now but I'm getting another error with a RequestTimeout that can be seen in the log. I'm not sure why it is getting a bad request, I tried with a .md file as well and ran into the same error.
aws_sdk_2018-06-29-02.log

[marcomagdy]

Probably because you’re hardcoding the content length and sending a file that is smaller.
The SDK will calculate the content length for you.

[chrishadiono]

Ok, I was getting an error before when I didn't set the content length
PutObject error: BadRequest Unable to parse ExceptionName: BadRequest Message: An error occurred when parsing the HTTP request.
and I'm getting that error again now still trying with .jpg, .md, and binary file types. It is getting the content-length at -1, which I'm guessing means that it is not getting the file? The file is in the same directory as where I'm running from so I don't know how that would be the case.
aws_sdk_2018-06-29-03.log

[marcomagdy]

Check the filestream after opening it and before using it.