Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boto3 resource ignored for s3 uploads #575

Closed
jackluo923 opened this issue Jan 3, 2021 · 0 comments · Fixed by #576
Closed

Boto3 resource ignored for s3 uploads #575

jackluo923 opened this issue Jan 3, 2021 · 0 comments · Fixed by #576

Comments

@jackluo923
Copy link
Contributor

Problem description

When specifying custom aws boto3 resource such as custom endpoint_url, the resource variable is ignored for uploads.
The expected result is to use the specified resource. The actual result is that the specified resource is ignored.

Steps/code to reproduce the problem

transport_params = {'session': s3_session, 'resource': s3_session.resource('s3', endpoint_url="http://<private_url>:9000"}
with open("s3://testing/abc.log", mode='wb', transport_params=transport_params) as out_file:
   file.write(b'abc\n')

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/smart_open/s3.py", line 721, in __init__
    self._mp = _retry_if_failed(partial)
  File "/usr/local/lib/python3.6/dist-packages/smart_open/s3.py", line 1003, in _retry_if_failed
    return partial()
  File "/usr/local/lib/python3.6/dist-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateMultipartUpload operation: The AWS Access Key Id you provided does not exist in our records.

Versions

Please provide the output of:

Linux-5.4.0-1029-aws-aarch64-with-Ubuntu-18.04-bionic
3.6.9 (default, Oct  8 2020, 12:12:24)
smart_open 4.1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant