Skip to content

Conversation

@mrshu
Copy link
Contributor

@mrshu mrshu commented May 19, 2020

Description of changes:

  • Currently awswrangler ensures that at least some boto3 Session is
    used when it calls AWS resources. However, since it creates a
    boto3.Session() when non-null boto3 Session is passed as a
    parameter, it completely sidesteps the ability of the user to set
    parameters to the boto3 session that gets used by default (via the
    boto3.setup_default_session function).

  • This commit ensures that the ensure_session function uses boto3's
    default session (reachable via boto3._get_default_session()) which
    then also allows awswrangler users to set default parameters of the
    boto3 Session it uses by default via boto3.setup_default_session().

Signed-off-by: mr.Shu mr@shu.io

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.


Relevant link: https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3.html#setup_default_session

@mrshu
Copy link
Contributor Author

mrshu commented May 19, 2020

@igorborgest I am not sure how would one test something like this but I'd be happy to incorporate some tests, if that was advisable.

Furthermore, if this ever makes it to master it may make sense to update the second part of the tutorial, so that users are aware that the parameters of the default session can also be set globally -- much like in the pre-1.0.0 versions of awswrangler.

Please do let me know if there is anything you'd like me to update here.

Thanks!

* Currently `awswrangler` ensures that at least some boto3 Session is
  used when it calls AWS resources. However, since it creates a
  `boto3.Session()` when non-null boto3 Session is passed as a
  parameter, it completely sidesteps the ability of the user to set
  parameters to the boto3 session that gets used by default (via the
  `boto3.setup_default_session` function).

* This commit ensures that the `ensure_session` function uses boto3's
  default session (reachable via `boto3._get_default_session()`) which
  then also allows `awswrangler` users to set default parameters of the
  boto3 Session it uses by default via `boto3.setup_default_session()`.

Signed-off-by: mr.Shu <mr@shu.io>
@mrshu mrshu force-pushed the mrshu/use-boto3-default-session branch from 1eea0ee to c8ea710 Compare May 19, 2020 17:48
@igorborgest igorborgest self-assigned this May 19, 2020
@igorborgest igorborgest added the enhancement New feature or request label May 19, 2020
@igorborgest
Copy link
Contributor

Hi @mrshu, thanks for the contribution.

It makes a lot of sense.

I would like to avoid access to external private methods as much as possible, but I can't see an alternative by now.

I will merge it on the dev branch and then write some tests to cover it.

@igorborgest igorborgest changed the base branch from master to dev May 19, 2020 18:55
@igorborgest igorborgest merged commit 144a8f4 into aws:dev May 19, 2020
igorborgest added a commit that referenced this pull request May 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants