Allow botocore loading from zip #1969
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
Codecov Report
@@ Coverage Diff @@
## develop #1969 +/- ##
===========================================
+ Coverage 92.73% 92.73% +<.01%
===========================================
Files 53 53
Lines 10293 10295 +2
===========================================
+ Hits 9545 9547 +2
Misses 748 748
Continue to review full report at Codecov.
|
|
@stealthycoin @joguSD please review |
philipkimmey
added a commit
to roverdotcom/botocore
that referenced
this pull request
May 13, 2020
Using pkg_resources allows for loading modules from a zip. Original author: Gábor Lipták <gliptak@gmail.com> See: - boto/boto3#1770 - boto#1969
philipkimmey
added a commit
to roverdotcom/botocore
that referenced
this pull request
May 14, 2020
philipkimmey
added a commit
to roverdotcom/botocore
that referenced
this pull request
May 14, 2020
philipkimmey
added a commit
to roverdotcom/botocore
that referenced
this pull request
May 14, 2020
|
Why was this closed? Is there a different ticket where support for loading |
|
@andrewdunai I closed my PR as committers didn't show interest |
|
@stealthycoin @joguSD @jamesls @nateprewitt It's a shame this is not getting enough attention. This functionality basically renders |
|
I've got the same problem that causes the error on AWS lambda when botocore is packaged within a zip file. I hope this issue could be improved. |
|
@gliptak AWS can take a tremendously long time to follow up on pull requests. If you don't want to keep your attention on a months-or-years long open pull request, I'd be happy to take it over for you. Unless it's left open, I'm sure AWS will ignore it forever. |
|
@kojiromike yes I do close my PRs not receiving attention after about a month. Feel free to resubmit. You might have better success |
kojiromike
added a commit
to kojiromike/botocore
that referenced
this pull request
Jul 12, 2021
When attempting to use botocore / boto3 in AWS EMR or PySpark, it's necessary to load from a zipfile. However, botocore does not support this.
```
+ PYTHONPATH=botocore.zip:boto3.zip
+ .venv/bin/python -c 'import boto3; client=boto3.client("rds")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/__init__.py", line 91, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/session.py", line 258, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 827, in create_client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 700, in _get_internal_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 924, in get_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 163, in create_default_resolver
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 132, in _wrapper
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 424, in load_data
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints
```
This is a re-submission of boto#1969 [with permission from @gliptak](boto#1969 (comment)), the original author.
|
Wow, it's been more than a year. |
kojiromike
added a commit
to kojiromike/botocore
that referenced
this pull request
Apr 18, 2022
When attempting to use botocore / boto3 in AWS EMR or PySpark, it's necessary to load from a zipfile. However, botocore does not support this.
```
+ PYTHONPATH=botocore.zip:boto3.zip
+ .venv/bin/python -c 'import boto3; client=boto3.client("rds")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/__init__.py", line 91, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/session.py", line 258, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 827, in create_client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 700, in _get_internal_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 924, in get_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 163, in create_default_resolver
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 132, in _wrapper
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 424, in load_data
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints
```
This is a re-submission of boto#1969 [with permission from @gliptak](boto#1969 (comment)), the original author.
kojiromike
added a commit
to kojiromike/botocore
that referenced
this pull request
Apr 18, 2022
When attempting to use botocore / boto3 in AWS EMR or PySpark, it's necessary to load from a zipfile. However, botocore does not support this.
```
+ PYTHONPATH=botocore.zip:boto3.zip
+ .venv/bin/python -c 'import boto3; client=boto3.client("rds")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/__init__.py", line 91, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/session.py", line 258, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 827, in create_client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 700, in _get_internal_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 924, in get_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 163, in create_default_resolver
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 132, in _wrapper
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 424, in load_data
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints
```
This is a re-submission of boto#1969 [with permission from @gliptak](boto#1969 (comment)), the original author.
kojiromike
added a commit
to kojiromike/botocore
that referenced
this pull request
May 18, 2022
When attempting to use botocore / boto3 in AWS EMR or PySpark, it's necessary to load from a zipfile. However, botocore does not support this.
```
+ PYTHONPATH=botocore.zip:boto3.zip
+ .venv/bin/python -c 'import boto3; client=boto3.client("rds")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/__init__.py", line 91, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/session.py", line 258, in client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 827, in create_client
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 700, in _get_internal_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 924, in get_component
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 163, in create_default_resolver
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 132, in _wrapper
File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 424, in load_data
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints
```
This is a re-submission of boto#1969 [with permission from @gliptak](boto#1969 (comment)), the original author.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Gábor Lipták gliptak@gmail.com
boto/boto3#1770