-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building functions that require native libraries to be present #973
Comments
Any idea when this will be prioritized? What's the expected cost of implementation? This is a hard blocker for me. If I can get some guidance on how to implement this so that it's mergeable, I'd like to take a stab at it. Barring that, we'll likely work around this by forking SAM in a hacky way. |
@clin88 What are you trying to do? This issue is for building against native libraries in things like a Layer. If you ware just trying to build against the native libraries on Lambda you can use |
Ya, building against native libraries is what I'm trying to solve. Namely, we use psycopg2cffi, which requires postgres, and pysaml2, which requires libxmlsec. And a few others I believe. |
I also needed to do something like this (wanted to |
@jfuss I heard this phrase many times |
@jfuss Our pipeline is already running as gitlab runner and i've been unsuccessful in using "--use-container" due to the docker in docker stuff. I just ran into another python module that has native C libraries and this issue has raised its ugly head. |
agree.
it would be nice if sam-cli allows us to create a layer which includes common dependencies and ref that arn in template. |
Something like this can be used as workaround |
@jfuss The problem here is that Is there currently a story for adding system packages (like postgres packages) to our lambda containers via SAM? How would one add a dependency on a system-level package? |
Is it possible to use a Dockerfile in conjunction with AWS SAM to augment the container used by a layer? |
This seems to be a breaking change from sam-cli v0.x to v1.x |
When I do
on this line from pdf2image import convert_from_bytes
pages = convert_from_bytes(pdf) This, however, works when I run the same thing from the Python console. What can I do? |
Following up on this issue, AWS SAM supports Image based Lambda functions. You could define a base image which has all the requisites libraries and then build your serverless function based on that. |
Outside of moving to the Closing this as there is now a way to get custom tools needed into the build image that can be linked against. If there is something still outstanding here, please create a new Github Issue and the team will take another look. |
|
Describe your idea/feature/enhancement
A number of the lambda languages have facilities for linking to C libraries for runtime performance. Ruby makes very heavy use of this, but it's not uncommon to see in Python or NodeJS as well. Even java is capable of it, though it's not all that common.
For example, in Ruby if you want to talk to a mysql database, you use a gem called mysql2 that expects to link against libmysqlclient.so. A layer is the natural place to store this .so file (rather than package it with your function code), but this doesn't solve the issue of having the development headers available at function build time.
Proposal
Provide a mechanism for injecting build-time dependencies into a build container as part of the sam build process so that native libraries can be compiled. Ideally this would also spit out a lambda layer with the runtime support libraries necessary for the function to actually run.
Things to consider:
Additional Details
I've recorded my work on this and posted a Dockerfile and a couple scripts that I'm using to do this manually outside of the sam build process now. I don't consider this to be a final solution or even necessarily a valid approach. It does demonstrate part of the problem I'm facing though.
https://github.com/jasonmk/lambda-gem-compiler
The text was updated successfully, but these errors were encountered: