Need to be clearer about installing Python packages with C++/DLL components

[athewsey]

The Working with Python > Deployment Package page section on "Updating a function with additional dependencies" misses an important point that can confuse users:

Following these instructions to pip install --target ./package on your build environment only works if either:

• Your target libraries are pure Python (or any bundled native dependencies are cross-platform), or
• Your build environment architecture is similar to the Lambda runtime.

For example (a typical request), I cannot pip install --target ./package Pillow on my Mac, bundle that to a Lambda deployment package upload it. My Lambda will give an error message something like:

{
    "errorMessage": "Unable to import module 'lambda_function': Cannot import name '_imaging' from 'PIL' (/var/task/PIL/__init__.py)",
    "errorType": "Runtime.ImportModuleError"
}

...If I'm deep on Python packaging, I might trace that this is because the line from . import _imaging as core buried in the PIL source code will work on my machine (where it resolves _imaging.cpython-38-darwin.so included in the bundle) but fail on Amazon Linux (because the architecture doesn't match).

If I'm not, it will probably take me a long time to debug as I'll be looking for solutions specific to my library & runtimes!

Please consider adding a caution on this (perhaps mentioning some popular libraries like PIL/Pillow which bundle architecture-specific components), and some discussion on workarounds such as:

• Creating the bundle (or at least the packaging part) on a compatible environment e.g. an EC2 instance, a SageMaker Notebook, etc)
• Using the AWS SAM CLI on your local machine with the --container option, so that SAM sets up a Docker container with a Lambda-like environment for you and simplifies the process.

Thanks!

[athewsey]

Whoops, GitHub had an error and looks like this is duplicated now - closing this one!