Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing 'boto3_session' in aws-data-wrangler/awswrangler/s3.py #172

Closed
hjuhel-cdpq opened this issue Apr 13, 2020 · 2 comments
Closed

Missing 'boto3_session' in aws-data-wrangler/awswrangler/s3.py #172

hjuhel-cdpq opened this issue Apr 13, 2020 · 2 comments
Assignees
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@hjuhel-cdpq
Copy link

Describe the bug
Using s3.to_parquetwith mode overwrite_partitions fails since the call to delete_objects does not propagate the boto3_session

To Reproduce

  • Try to create a dataset while providing a boto3 session
import awswrangler as wr
import pandas as pd
import boto3

# The custom boto3 session 
sess = boto3.Session(aws_access_key_id='<foo>', aws_secret_access_key='<bar>')

# The dataframe to write
df = pd.DataFrame({'one': [0,], 'two': ["spam"], 'three': [True]})

wr.s3.to_parquet(
    df=df,
    path='s3://path',
    dataset=True,
    mode="overwrite_partitions",
    partition_cols=["two"],
    boto3_session=sess
)

The previous snipet results in an authentication error, probably because of the missing boto3_session parameter in the delete call:

if mode == "overwrite_partitions":
    delete_objects(path=prefix, use_threads=use_threads
@hjuhel-cdpq hjuhel-cdpq added the bug Something isn't working label Apr 13, 2020
@igorborgest igorborgest added the good first issue Good for newcomers label Apr 13, 2020
@igorborgest
Copy link
Contributor

Hi @hjuhel-cdpq! Thank you very much for report it.

Will be fixed and released later today in the version 1.0.2.

igorborgest added a commit that referenced this issue Apr 13, 2020
@igorborgest
Copy link
Contributor

Thanks @hjuhel-cdpq, this issue is resolved on version 1.0.2. Please, give feedback when possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants