Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws s3api create-bucket not honoring --region argument #1667

Closed
timkendrick opened this issue Dec 4, 2015 · 4 comments
Closed

aws s3api create-bucket not honoring --region argument #1667

timkendrick opened this issue Dec 4, 2015 · 4 comments
Labels
closing-soon This issue will automatically close in 4 days unless further comments are made. guidance Question that needs advice or information. s3api service-api This issue is due to a problem in a service API, not the SDK implementation.

Comments

@timkendrick
Copy link

I've just started using the AWS CLI, and it seems that whatever I try to create a bucket using the aws s3api create-bucket command, it ignores both the --region argument passed to the command, and the region setting in my ~/.aws/config.

Example output below:

$ aws --version
aws-cli/1.9.11 Python/2.7.10 Darwin/14.5.0 botocore/1.3.11
$ aws s3api create-bucket --bucket bd354a50-2ec4-4b5c-b919-30f0581d65a3 --region eu-west-1
{
    "Location": "/bd354a50-2ec4-4b5c-b919-30f0581d65a3"
}
$ aws s3api get-bucket-location --bucket bd354a50-2ec4-4b5c-b919-30f0581d65a3
{
    "LocationConstraint": null
}

...however it all works fine when I use the aws s3 mb command:

$ aws s3 mb s3://158d3cbb-97fa-43af-a637-992b4e80dd57 --region eu-west-1
make_bucket: s3://158d3cbb-97fa-43af-a637-992b4e80dd57/
$ aws s3api get-bucket-location --bucket 158d3cbb-97fa-43af-a637-992b4e80dd57
{
    "LocationConstraint": "eu-west-1"
}

Full debug output below:

$ aws s3api create-bucket --bucket bd354a50-2ec4-4b5c-b919-30f0581d65a3 --region eu-west-1 --debug
2015-12-04 15:37:16,085 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.9.9 Python/2.7.10 Darwin/14.5.0 botocore/1.3.9
2015-12-04 15:37:16,086 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3api', 'create-bucket', '--bucket', 'bd354a50-2ec4-4b5c-b919-30f0581d65a3', '--region', 'eu-west-1', '--debug']
2015-12-04 15:37:16,086 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x107d588c0>
2015-12-04 15:37:16,086 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider at 0x107d28758>
2015-12-04 15:37:16,086 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2015-12-04 15:37:16,118 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler <function register_retries_for_service at 0x1078140c8>
2015-12-04 15:37:16,118 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: s3
2015-12-04 15:37:16,120 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3api: calling handler <function add_waiters at 0x107d34c80>
2015-12-04 15:37:16,123 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'acl', <awscli.arguments.CLIArgument object at 0x1082b96d0>), (u'bucket', <awscli.arguments.CLIArgument object at 0x1082b9710>), (u'create-bucket-configuration', <awscli.arguments.CLIArgument object at 0x1082b9750>), (u'grant-full-control', <awscli.arguments.CLIArgument object at 0x1082b9790>), (u'grant-read', <awscli.arguments.CLIArgument object at 0x1082b97d0>), (u'grant-read-acp', <awscli.arguments.CLIArgument object at 0x1082b9810>), (u'grant-write', <awscli.arguments.CLIArgument object at 0x1082b9850>), (u'grant-write-acp', <awscli.arguments.CLIArgument object at 0x1082b9890>)])
2015-12-04 15:37:16,124 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.create-bucket: calling handler <function add_streaming_output_arg at 0x107acab90>
2015-12-04 15:37:16,124 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.create-bucket: calling handler <function add_cli_input_json at 0x107d1fed8>
2015-12-04 15:37:16,124 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.create-bucket: calling handler <function unify_paging_params at 0x107bf1848>
2015-12-04 15:37:16,127 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.s3api.create-bucket: calling handler <function add_generate_skeleton at 0x107d28398>
2015-12-04 15:37:16,127 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.create-bucket: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x1082b98d0>>
2015-12-04 15:37:16,127 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.s3api.create-bucket: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x1082b9910>>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.acl: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.bucket: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.s3.create-bucket: calling handler <awscli.argprocess.ParamShorthand object at 0x107d5bcd0>
2015-12-04 15:37:16,129 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'bd354a50-2ec4-4b5c-b919-30f0581d65a3' for parameter "bucket": u'bd354a50-2ec4-4b5c-b919-30f0581d65a3'
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.create-bucket-configuration: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.grant-full-control: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.grant-read: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.grant-read-acp: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.grant-write: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,129 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.grant-write-acp: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,130 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.cli-input-json: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,130 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.s3.create-bucket.generate-cli-skeleton: calling handler <function uri_param at 0x107ab3758>
2015-12-04 15:37:16,130 - MainThread - botocore.hooks - DEBUG - Event calling-command.s3api.create-bucket: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x1082b9910>>
2015-12-04 15:37:16,130 - MainThread - botocore.hooks - DEBUG - Event calling-command.s3api.create-bucket: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x1082b98d0>>
2015-12-04 15:37:16,130 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2015-12-04 15:37:16,130 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2015-12-04 15:37:16,130 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2015-12-04 15:37:16,130 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2015-12-04 15:37:16,132 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2015-12-04 15:37:16,135 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x1077fc938>
2015-12-04 15:37:16,135 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x1077e98c0>
2015-12-04 15:37:16,181 - MainThread - botocore.endpoint - DEBUG - Setting s3 timeout as (60, 60)
2015-12-04 15:37:16,181 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.s3.CreateBucket: calling handler <function validate_bucket_name at 0x107813de8>
2015-12-04 15:37:16,184 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.CreateBucket: calling handler <function add_expect_header at 0x1078142a8>
2015-12-04 15:37:16,184 - MainThread - botocore.endpoint - DEBUG - Making request for <botocore.model.OperationModel object at 0x1082d1490> (verify_ssl=True) with params: {'body': '', 'url': u'https://s3-eu-west-1.amazonaws.com/bd354a50-2ec4-4b5c-b919-30f0581d65a3', 'headers': {'User-Agent': 'aws-cli/1.9.9 Python/2.7.10 Darwin/14.5.0 botocore/1.3.9'}, 'query_string': {}, 'url_path': u'/bd354a50-2ec4-4b5c-b919-30f0581d65a3', 'method': u'PUT'}
2015-12-04 15:37:16,184 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.CreateBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x1082fe550>>
2015-12-04 15:37:16,185 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.CreateBucket: calling handler <function fix_s3_host at 0x1076be758>
2015-12-04 15:37:16,185 - MainThread - botocore.utils - DEBUG - Checking for DNS compatible bucket for: https://s3-eu-west-1.amazonaws.com/bd354a50-2ec4-4b5c-b919-30f0581d65a3
2015-12-04 15:37:16,185 - MainThread - botocore.utils - DEBUG - URI updated to: https://bd354a50-2ec4-4b5c-b919-30f0581d65a3.s3.amazonaws.com/
2015-12-04 15:37:16,185 - MainThread - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2015-12-04 15:37:16,186 - MainThread - botocore.auth - DEBUG - HTTP request method: PUT
2015-12-04 15:37:16,186 - MainThread - botocore.auth - DEBUG - StringToSign:
PUT


Fri, 04 Dec 2015 15:37:16 GMT
/bd354a50-2ec4-4b5c-b919-30f0581d65a3/
2015-12-04 15:37:16,191 - MainThread - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [PUT]>
2015-12-04 15:37:16,192 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): bd354a50-2ec4-4b5c-b919-30f0581d65a3.s3.amazonaws.com
2015-12-04 15:37:17,280 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "PUT / HTTP/1.1" 200 0
2015-12-04 15:37:17,281 - MainThread - botocore.parsers - DEBUG - Response headers: {'content-length': '0', 'x-amz-id-2': 'OE2fJyBraV1BC2Fkzg6k52ModDDCw42Z+9X7Z2xvYuwAx5Z9mIGRrCA/WdVoyGRvzxvqui91qBQ=', 'server': 'AmazonS3', 'x-amz-request-id': '7F1144E54622C18C', 'location': '/bd354a50-2ec4-4b5c-b919-30f0581d65a3', 'date': 'Fri, 04 Dec 2015 15:37:18 GMT'}
2015-12-04 15:37:17,281 - MainThread - botocore.parsers - DEBUG - Response body:

2015-12-04 15:37:17,281 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.CreateBucket: calling handler <botocore.retryhandler.RetryHandler object at 0x1082a20d0>
2015-12-04 15:37:17,282 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2015-12-04 15:37:17,282 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.CreateBucket: calling handler <function enhance_error_msg at 0x107d1fc80>
2015-12-04 15:37:17,282 - MainThread - botocore.hooks - DEBUG - Event after-call.s3.CreateBucket: calling handler <awscli.errorhandler.ErrorHandler object at 0x107d5bd90>
2015-12-04 15:37:17,282 - MainThread - awscli.errorhandler - DEBUG - HTTP Response Code: 200
2015-12-04 15:37:17,283 - MainThread - awscli.formatter - DEBUG - RequestId: 7F1144E54622C18C
{
    "Location": "/bd354a50-2ec4-4b5c-b919-30f0581d65a3"
}
@kyleknap
Copy link
Member

kyleknap commented Dec 8, 2015

So I would recommend using aws s3 mb here to make your bucket. For the s3api create-bucket command, you need to specify a LocationConstraint to have the bucket exist in that location:

$ aws s3api create-bucket --bucket mybucketfoo6 --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1 
{
    "Location": "http://mybucketfoo6.s3.amazonaws.com/"
}
$ aws s3api get-bucket-location --bucket mybucketfoo6
{
    "LocationConstraint": "eu-west-1"
}

I recommend using aws s3 mb because it autofills the LocationConstraint for you. Let me know if you have any more questions.

@kyleknap kyleknap added closing-soon This issue will automatically close in 4 days unless further comments are made. s3api api-question labels Dec 8, 2015
@timkendrick
Copy link
Author

Yep, I've been using aws s3 mb since figuring this out.

I just thought I should probably report it, seeing as I ended up wasting far too long trying to figure out why the example in the documentation wasn't working for me – perhaps the documentation needs updating?

@kyleknap
Copy link
Member

kyleknap commented Dec 8, 2015

Makes sense. I will add an example to reflect that.

@timkendrick
Copy link
Author

Great, thanks for the help!

@kyleknap kyleknap mentioned this issue Dec 8, 2015
Merged
timothyjosefik pushed a commit to timothyjosefik/kops that referenced this issue Sep 20, 2017
Changed the s3 bucket creation command. This was done because if some…
e296412 
…one is following along but wants to use a different region then the original command will not always create the bucket in that region. Here is the issue in the aws-cli that provides more information on the problem. aws/aws-cli#1667
@timothyjosefik timothyjosefik mentioned this issue Sep 20, 2017
Merged
timothyjosefik added a commit to timothyjosefik/kops that referenced this issue Sep 20, 2017
@timothyjosefik
Changed the s3 bucket creation command. This was done because if some…
18129f4 
…one is following along but wants to use a different region then the original command will not always create the bucket in that region. Here is the issue in the aws-cli that provides more information on the problem. aws/aws-cli#1667
@diehlaws diehlaws added guidance Question that needs advice or information. service-api This issue is due to a problem in a service API, not the SDK implementation. and removed api-question labels Jan 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
closing-soon This issue will automatically close in 4 days unless further comments are made. guidance Question that needs advice or information. s3api service-api This issue is due to a problem in a service API, not the SDK implementation.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@timkendrick @kyleknap @diehlaws