New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Module initialization error in AWS Lambda #52

Closed
tmilicic opened this Issue Jul 29, 2017 · 5 comments

Comments

Projects
None yet
4 participants
@tmilicic

tmilicic commented Jul 29, 2017

When trying to use snappy inside AWS Lambda I am getting:
module initialization error: [Errno 2] No such file or directory: '/var/task/__pycache__/_cffi__x762f05ffx6bf5342b.c'

@martindurant

This comment has been minimized.

Show comment
Hide comment
@martindurant

martindurant Jul 31, 2017

Collaborator

I'm afraid I know nothing about installation in Lambda, why it might be different from other architectures. With luck, other people will be able to comment.

Collaborator

martindurant commented Jul 31, 2017

I'm afraid I know nothing about installation in Lambda, why it might be different from other architectures. With luck, other people will be able to comment.

@tmilicic

This comment has been minimized.

Show comment
Hide comment
@tmilicic

tmilicic Jul 31, 2017

Ok thank you, I will no longer need this since I am just gonna use EC2 instance but it would be interesting to see why is this happening.

tmilicic commented Jul 31, 2017

Ok thank you, I will no longer need this since I am just gonna use EC2 instance but it would be interesting to see why is this happening.

@andrix

This comment has been minimized.

Show comment
Hide comment
@andrix

andrix Aug 7, 2017

Owner

@tmilicic I don't have the opportunity to use AWS Lambda to debug it, if you have the chance you can post the results here. I believe that others can help too, with more debug information.

In the meantime, I will close the issue until having further information.

Owner

andrix commented Aug 7, 2017

@tmilicic I don't have the opportunity to use AWS Lambda to debug it, if you have the chance you can post the results here. I believe that others can help too, with more debug information.

In the meantime, I will close the issue until having further information.

@andrix andrix closed this Aug 7, 2017

@pez5001

This comment has been minimized.

Show comment
Hide comment
@pez5001

pez5001 Nov 7, 2017

I'm also running into this issue in AWS Lambda, managed to isolate it to just the snappy import.

This is an AWS Lambda function running python 3.6, with python-snappy==0.5.1 installed.
Interestingly, I can't reproduce it on the amazon linux instance we use to build the lambda package, (including when I set the working directory to read-only), but only when I deploy it and invoke it in Lambda.
I was able to hack in to my entry point a way to print the exception out to cloudwatch (otherwise the bug blows up the process before logging can even get set up) so please excuse the formatting, but here's the exception that gets raised when I try to call import snappy

[Errno 30] Read-only file system: '/var/task/__pycache__/_cffi__x70d2a14ex6bf5342b.c'
Traceback (most recent call last):
File "/var/task/snappy.py", line 47, in <module>
from _snappy import UncompressError, compress, decompress, \
ImportError: libsnappy.so.1: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/task/lambda_handler.py", line 9, in lambda_handler
import snappy
File "/var/task/snappy.py", line 50, in <module>
from snappy_cffi import UncompressError, compress, decompress, \
File "/var/task/snappy_cffi.py", line 169, in <module>
''', libraries=["snappy"])
File "/var/task/cffi/api.py", line 444, in verify
lib = self.verifier.load_library()
File "/var/task/cffi/verifier.py", line 103, in load_library
self._write_source()
File "/var/task/cffi/verifier.py", line 192, in _write_source
with open(self.sourcefilename, "w") as fp:
OSError: [Errno 30] Read-only file system: '/var/task/__pycache__/_cffi__x70d2a14ex6bf5342b.c'

So it looks like cffi is trying to write out that c file into the read-only directory the Lambda function gets run in, which is the source of the permission denied error.

What's especially interesting to me is I've been using snappy successfully in Lambda for a while, with an existing function that's python-snappy==0.5 in python 2.7.

pez5001 commented Nov 7, 2017

I'm also running into this issue in AWS Lambda, managed to isolate it to just the snappy import.

This is an AWS Lambda function running python 3.6, with python-snappy==0.5.1 installed.
Interestingly, I can't reproduce it on the amazon linux instance we use to build the lambda package, (including when I set the working directory to read-only), but only when I deploy it and invoke it in Lambda.
I was able to hack in to my entry point a way to print the exception out to cloudwatch (otherwise the bug blows up the process before logging can even get set up) so please excuse the formatting, but here's the exception that gets raised when I try to call import snappy

[Errno 30] Read-only file system: '/var/task/__pycache__/_cffi__x70d2a14ex6bf5342b.c'
Traceback (most recent call last):
File "/var/task/snappy.py", line 47, in <module>
from _snappy import UncompressError, compress, decompress, \
ImportError: libsnappy.so.1: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/task/lambda_handler.py", line 9, in lambda_handler
import snappy
File "/var/task/snappy.py", line 50, in <module>
from snappy_cffi import UncompressError, compress, decompress, \
File "/var/task/snappy_cffi.py", line 169, in <module>
''', libraries=["snappy"])
File "/var/task/cffi/api.py", line 444, in verify
lib = self.verifier.load_library()
File "/var/task/cffi/verifier.py", line 103, in load_library
self._write_source()
File "/var/task/cffi/verifier.py", line 192, in _write_source
with open(self.sourcefilename, "w") as fp:
OSError: [Errno 30] Read-only file system: '/var/task/__pycache__/_cffi__x70d2a14ex6bf5342b.c'

So it looks like cffi is trying to write out that c file into the read-only directory the Lambda function gets run in, which is the source of the permission denied error.

What's especially interesting to me is I've been using snappy successfully in Lambda for a while, with an existing function that's python-snappy==0.5 in python 2.7.

@pez5001

This comment has been minimized.

Show comment
Hide comment
@pez5001

pez5001 Nov 7, 2017

Actually, it looks like this isn't a bug with this library at all, rather a quirk of implementing in Lambda.

This line in the error output is the clue:
ImportError: libsnappy.so.1: cannot open shared object file: No such file or directory

Including that library in the root of the code package deployed to Lambda solved the issue - it must be that if that's not present, cffi will try to build it itself.

pez5001 commented Nov 7, 2017

Actually, it looks like this isn't a bug with this library at all, rather a quirk of implementing in Lambda.

This line in the error output is the clue:
ImportError: libsnappy.so.1: cannot open shared object file: No such file or directory

Including that library in the root of the code package deployed to Lambda solved the issue - it must be that if that's not present, cffi will try to build it itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment