awswrangler doesn't work with moto

[vincentclaes]

moto is a popular library to mock aws services.
all the calls that happen via boto3 are mocked by moto so that you can test locally without any connection to real aws infrastructure.

when I write parquet files to s3 in my tests, it works without any error. but when i want to do assertions on the objects i have written to S3, none can be found.

any idea how this can happen? i assume awswrangler uses boto3 to communicate with s3?

[igorborgest]

Thank you for reaching out @vincentclaes!

Starting on version 1.0.0 Wrangler can run on moto with functionalities related with Amazon S3 and Amazon EMR (partially).

I also added some test cases for Wrangler to ensure that the moto compatibility will be kept. (PR above).

[lu-liu-rft]

Thank you for reaching out @vincentclaes!

Starting on version 1.0.0 Wrangler can run on moto with functionalities related with Amazon S3 and Amazon EMR (partially).

I also added some test cases for Wrangler to ensure that the moto compatibility will be kept. (PR above).

Hi the link for 'test cases' is invalid, could you take a look please? Thanks.

[jaidisido]

@lu-liu-rft : https://github.com/awslabs/aws-data-wrangler/blob/main/tests/test_moto.py

[lu-liu-rft]

@lu-liu-rft : https://github.com/awslabs/aws-data-wrangler/blob/main/tests/test_moto.py

Thanks!

[nareshNishad]

I tried to use some of the example specified in the link but they all goes to internet not testing them locally. Is there any way to test them locally, without hitting actual AWS.