Skip to content
This repository has been archived by the owner on Jun 15, 2024. It is now read-only.

Python Layers using other layers packages #153

Closed
ayush-mehta opened this issue Jan 17, 2024 · 3 comments
Closed

Python Layers using other layers packages #153

ayush-mehta opened this issue Jan 17, 2024 · 3 comments

Comments

@ayush-mehta
Copy link

ayush-mehta commented Jan 17, 2024

Using serverless-layers@2.7
Getting the same issue mentioned in #114
@agutoli you mentioned that you made some changes in version 2.7 so it should work, but I am facing the same issue, here is my serverless.yml file

  serverless-layers:
    - numpy-pandas:
        layersDeploymentBucket: clara-lambda-layers
        dependenciesPath: layers/numpy-pandas/requirements.txt
    - pdf2image-deskew:
        layersDeploymentBucket: clara-lambda-layers
        functions:
          - PytesseractFetchOCR
        dependenciesPath: layers/pdfPreprocessing/requirements.txt
    - pdfplumber:
        layersDeploymentBucket: clara-lambda-layers
        functions:
          - PdfPlumberFetchOCR
        dependenciesPath: layers/pdfplumber/requirements.txt
    - pytesseract:
        layersDeploymentBucket: clara-lambda-layers
        functions:
          - PytesseractFetchOCR
        dependenciesPath: layers/pytesseract/requirements.txt```
@agutoli
Copy link
Owner

agutoli commented Jan 18, 2024

@ayush-mehta You are using OpenCV, Pandas, NumPy, and other large libraries, which are probably exceeding the Lambda limit/quota.
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

@ayush-mehta
Copy link
Author

I agree that the libraries are large that is why I am breaking them into multiple layers, I have checked individually that each layer's requirements are less then 250MB individually. But the thing that is happening here is that the plugin is appending the requirements of the second layer into the first one.
I tried it out and checked in s3 bucket where the zip files are stored, it created the first layer perfectly, but when installing the second layer, it again installed the first layer's requirement into the same folder, so I believe the issue first described in #114 is recurring here.
Correct me if I am wrong anywhere @agutoli

agutoli pushed a commit that referenced this issue Mar 8, 2024
@agutoli
Copy link
Owner

agutoli commented Mar 8, 2024

@ayush-mehta sorry for late reply (really busy with my job). I've published a bugfix:

serverless-layers@2.8.3

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants