aws_s3_file doesn't download file from S3, just creates blank file when IAM role is assigned to instane

[lkjangir]

Cookbook version

[Version of the cookbook where you are encountering the issue]
7.1.2

Chef-client version

[Version of chef-client in your environment]
13.3.42

Platform Details

[Operating system distribution and release version. Cloud provider if running in the cloud]
Amazon Linux

Scenario:

[What you are trying to achieve and you can't?]
EC2 instance has IAM role assigned and I just want to download the file from S3 bucket, it creates a blank file, doesn't download.

Steps to Reproduce:

[If you are filing an issue what are the things we need to do in order to repro your problem? How are you using this cookbook or any resources it includes?]
I have this code in my recipe...

include_recipe 'aws'
aws_s3_file "/home/kern2.log" do
  bucket "project-req-software"
  remote_path "kern.log"
#  aws_assume_role_arn 'arn:aws:iam::xxxxxxxxxxxxx:role/EC2'
#  aws_role_session_name 'temp-session'
end

Expected Result:

It should download the file, but it doesn't

Actual Result:

it doesn't give any error, chef-client completes process successfully.

[iennae]

@lkjangir This is really hard to debug based on the information you've provided. Based on the tests that run for this cookbook when

  aws_access_key node['aws_test']['key_id']
  aws_secret_access_key node['aws_test']['access_key']

are defined, the cookbook converges successfully and the file is downloaded successfully:

-rw-r--r--  1 root     root     173147 Sep 12 00:32 a_file
-rw-r--r--  1 root     root         18 Sep 12 00:32 a_file_2

If the keys aren't correct or missing the converge fails. Are you sure the file isn't empty? No one else has reported an issue which leads me to think that there might be a problem in your environment.

[iennae]

Please reopen if you are still having problems and can update with more information please reopen. Thanks.

[rbreckenridge]

@lkjangir did you resolve this issue? I'm running into the exact same thing....

[jonathanan]

I just ran into this. Double check the region of your bucket and what you have configured in aws_s3_file.

@rbreckenridge

[iennae]

@jonathanan thanks for the additional context to how to possibly debug.

[voroniys]

I'm having similar issue when using asw_s3_file on instance running in different region and when region is not specified in the resource.
Normally speaking region specification is not needed - for instance aws cli working properly no matter which region is S3 bucket and instance. But aws_s3_file fails if region is not specified and EC2 instance is in different region.

[sglajchEG]

I think the region requirement for the aws_s3_file command should be removed. It's not a given that your s3 bucket should be in the same region as the EC2 instance. I think a common scenario for grabbing S3 files during a chef run would be to pull down common configuration files/rpm packages/etc, most or all of which aren't region specific things. S3 doesn't require a region to download anyways. You can aws s3api get-object on the machine itself by just specifying the bucket and object path, no region required.