The Sentry Python serverless SDK docker layer doesn't seem to work

[Makeshift]

Environment

How do you use Sentry?
On-premise (21.10.0 0d35c1b)

Which SDK and version?
python-serverless-sdk:6 (Container layer)
sentry-sdk 1.5.0 (Python)

All of these tests ran locally in a Docker container, though the same happens in an actual AWS lambda.

Steps to Reproduce

Lambda Handler (With docker layer)

Given this dockerfile:

FROM public.ecr.aws/lambda/python:3.9

COPY --from=public.ecr.aws/sentry/sentry-python-serverless-sdk:6 /opt/ /opt

COPY main.py ${LAMBDA_TASK_ROOT}

ENV PYTHONPATH=${LAMBDA_RUNTIME_DIR}:/opt/python
ENV SENTRY_INITIAL_HANDLER="main.handler"
ENV SENTRY_TRACES_SAMPLE_RATE="1.0"
ENV SENTRY_DEBUG=1
ENV SENTRY_DSN=https://<snip>@sentry.the-iea.tools/5

CMD ["sentry_sdk.integrations.init_serverless_sdk.sentry_lambda_handler"]

and code:

def handler(event, context):
  """Entry point for Lambda."""
  print(event)
  division_by_zero = 1 / 0

As detailed in the tutorial

In Code (Dockerfile layer)

I have also tried importing Sentry directly in the code, as so:

FROM public.ecr.aws/lambda/python:3.9

COPY --from=public.ecr.aws/sentry/sentry-python-serverless-sdk:6 /opt/ /opt

COPY main.py ${LAMBDA_TASK_ROOT}

ENV PYTHONPATH=${LAMBDA_RUNTIME_DIR}:/opt/python
ENV SENTRY_DSN=https://<snip>@sentry.the-iea.tools/5

CMD ["main.handler"]

import os
import socket
import sentry_sdk

print(os.environ['SENTRY_DSN'])
sentry_sdk.init(
  dsn=os.environ['SENTRY_DSN'],
  traces_sample_rate=1,
  debug=True
)

def handler(event, context):
  """Entry point for Lambda."""
  print(event)
  division_by_zero = 1 / 0

Not using the official AWS container as a base

FROM python:3.9

COPY --from=public.ecr.aws/sentry/sentry-python-serverless-sdk:6 /opt/ /opt

RUN pip3 install awslambdaric
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie

COPY main.py entry.sh ./

RUN chmod +x /usr/bin/aws-lambda-rie entry.sh

ENV SENTRY_DEBUG=1
ENV SENTRY_DSN=https://<snip>@sentry.the-iea.tools/5
ENV SENTRY_INITIAL_HANDLER=main.handler
ENV SENTRY_TRACES_SAMPLE_RATE=1.0
ENV PYTHONPATH=$PYTHONPATH:/opt/python

ENTRYPOINT ["/bin/sh", "entry.sh"]
CMD ["sentry_sdk.integrations.init_serverless_sdk.sentry_lambda_handler"]

Expected Result

Sentry would catch the error and forward it.

Actual Result

Lambda Handler (With docker layer)

Logs:

$ docker run -it -p 9000:8080 test
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
INFO[0002] extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
WARN[0002] Cannot list external agents                   error="open /opt/extensions: no such file or directory"
START RequestId: a0ad98df-5101-4fe3-b3bb-1526fad2b3d0 Version: $LATEST
https://<snip>@sentry.the-iea.tools/5
 [sentry] DEBUG: Setting up integrations (with default = True)
[DEBUG] 2021-11-25T18:53:11.59Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Setting up integrations (with default = True)
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.django.DjangoIntegration: Django not installed
[DEBUG] 2021-11-25T18:53:11.61Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.django.DjangoIntegration: Django not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.flask.FlaskIntegration: Flask is not installed
[DEBUG] 2021-11-25T18:53:11.62Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.flask.FlaskIntegration: Flask is not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.bottle.BottleIntegration: Bottle not installed
[DEBUG] 2021-11-25T18:53:11.62Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.bottle.BottleIntegration: Bottle not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.falcon.FalconIntegration: Falcon not installed
[DEBUG] 2021-11-25T18:53:11.63Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.falcon.FalconIntegration: Falcon not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.sanic.SanicIntegration: Sanic not installed
[DEBUG] 2021-11-25T18:53:11.64Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.sanic.SanicIntegration: Sanic not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.celery.CeleryIntegration: Celery not installed
[DEBUG] 2021-11-25T18:53:11.64Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.celery.CeleryIntegration: Celery not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.rq.RqIntegration: RQ not installed
[DEBUG] 2021-11-25T18:53:11.65Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.rq.RqIntegration: RQ not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.aiohttp.AioHttpIntegration: AIOHTTP not installed
[DEBUG] 2021-11-25T18:53:11.66Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.aiohttp.AioHttpIntegration: AIOHTTP not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.tornado.TornadoIntegration: Tornado not installed
[DEBUG] 2021-11-25T18:53:11.66Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.tornado.TornadoIntegration: Tornado not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.sqlalchemy.SqlalchemyIntegration: SQLAlchemy not installed.
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Did not import default integration sentry_sdk.integrations.sqlalchemy.SqlalchemyIntegration: SQLAlchemy not installed.
 [sentry] DEBUG: Enabling integration logging
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration logging
 [sentry] DEBUG: Enabling integration stdlib
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration stdlib
 [sentry] DEBUG: Enabling integration excepthook
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration excepthook
 [sentry] DEBUG: Enabling integration dedupe
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration dedupe
 [sentry] DEBUG: Enabling integration atexit
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration atexit
 [sentry] DEBUG: Enabling integration modules
[DEBUG] 2021-11-25T18:53:11.67Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration modules
 [sentry] DEBUG: Enabling integration argv
[DEBUG] 2021-11-25T18:53:11.68Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration argv
 [sentry] DEBUG: Enabling integration threading
[DEBUG] 2021-11-25T18:53:11.68Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration threading
 [sentry] DEBUG: Enabling integration boto3
[DEBUG] 2021-11-25T18:53:11.68Z a0ad98df-5101-4fe3-b3bb-1526fad2b3d0    Enabling integration boto3
{'payload': 'hello world!'}
    division_by_zero = 1 / 0line 15, in handlervent, context)dk/__init__.py", line 77, in sentry_lambda_handler
END RequestId: a0ad98df-5101-4fe3-b3bb-1526fad2b3d0
REPORT RequestId: a0ad98df-5101-4fe3-b3bb-1526fad2b3d0  Init Duration: 0.13 ms  Duration: 272.94 ms     Billed Duration: 273 ms Memory Size: 3008 MB    Max Memory Used: 3008 MB
^CINFO[0006] Received signal                               signal=interrupt

In code (Dockerfile layer)

Logs:

$ docker run -it -p 9000:8080 test
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
INFO[0002] extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
WARN[0002] Cannot list external agents                   error="open /opt/extensions: no such file or directory"
START RequestId: 83306b75-90b7-4f09-9cae-326af9dfe531 Version: $LATEST
https://<snip>@sentry.the-iea.tools/5
 [sentry] DEBUG: Setting up integrations (with default = True)
[DEBUG] 2021-11-25T18:56:16.907Z                Setting up integrations (with default = True)
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.django.DjangoIntegration: Django not installed
[DEBUG] 2021-11-25T18:56:16.921Z                Did not import default integration sentry_sdk.integrations.django.DjangoIntegration: Django not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.flask.FlaskIntegration: Flask is not installed
[DEBUG] 2021-11-25T18:56:16.923Z                Did not import default integration sentry_sdk.integrations.flask.FlaskIntegration: Flask is not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.bottle.BottleIntegration: Bottle not installed
[DEBUG] 2021-11-25T18:56:16.925Z                Did not import default integration sentry_sdk.integrations.bottle.BottleIntegration: Bottle not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.falcon.FalconIntegration: Falcon not installed
[DEBUG] 2021-11-25T18:56:16.926Z                Did not import default integration sentry_sdk.integrations.falcon.FalconIntegration: Falcon not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.sanic.SanicIntegration: Sanic not installed
[DEBUG] 2021-11-25T18:56:16.928Z                Did not import default integration sentry_sdk.integrations.sanic.SanicIntegration: Sanic not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.celery.CeleryIntegration: Celery not installed
[DEBUG] 2021-11-25T18:56:16.930Z                Did not import default integration sentry_sdk.integrations.celery.CeleryIntegration: Celery not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.rq.RqIntegration: RQ not installed
[DEBUG] 2021-11-25T18:56:16.931Z                Did not import default integration sentry_sdk.integrations.rq.RqIntegration: RQ not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.aiohttp.AioHttpIntegration: AIOHTTP not installed
[DEBUG] 2021-11-25T18:56:16.944Z                Did not import default integration sentry_sdk.integrations.aiohttp.AioHttpIntegration: AIOHTTP not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.tornado.TornadoIntegration: Tornado not installed
[DEBUG] 2021-11-25T18:56:16.946Z                Did not import default integration sentry_sdk.integrations.tornado.TornadoIntegration: Tornado not installed
 [sentry] DEBUG: Did not import default integration sentry_sdk.integrations.sqlalchemy.SqlalchemyIntegration: SQLAlchemy not installed.
[DEBUG] 2021-11-25T18:56:16.947Z                Did not import default integration sentry_sdk.integrations.sqlalchemy.SqlalchemyIntegration: SQLAlchemy not installed.
 [sentry] DEBUG: Setting up previously not enabled integration logging
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration logging
 [sentry] DEBUG: Setting up previously not enabled integration stdlib
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration stdlib
 [sentry] DEBUG: Setting up previously not enabled integration excepthook
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration excepthook
 [sentry] DEBUG: Setting up previously not enabled integration dedupe
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration dedupe
 [sentry] DEBUG: Setting up previously not enabled integration atexit
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration atexit
 [sentry] DEBUG: Setting up previously not enabled integration modules
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration modules
 [sentry] DEBUG: Setting up previously not enabled integration argv
[DEBUG] 2021-11-25T18:56:16.991Z                Setting up previously not enabled integration argv
 [sentry] DEBUG: Setting up previously not enabled integration threading
[DEBUG] 2021-11-25T18:56:16.992Z                Setting up previously not enabled integration threading
 [sentry] DEBUG: Setting up previously not enabled integration boto3
[DEBUG] 2021-11-25T18:56:16.992Z                Setting up previously not enabled integration boto3
 [sentry] DEBUG: Enabling integration logging
[DEBUG] 2021-11-25T18:56:16.992Z                Enabling integration logging
 [sentry] DEBUG: Enabling integration stdlib
[DEBUG] 2021-11-25T18:56:16.992Z                Enabling integration stdlib
 [sentry] DEBUG: Enabling integration excepthook
[DEBUG] 2021-11-25T18:56:16.992Z                Enabling integration excepthook
 [sentry] DEBUG: Enabling integration dedupe
[DEBUG] 2021-11-25T18:56:16.992Z                Enabling integration dedupe
 [sentry] DEBUG: Enabling integration atexit
[DEBUG] 2021-11-25T18:56:16.993Z                Enabling integration atexit
 [sentry] DEBUG: Enabling integration modules
[DEBUG] 2021-11-25T18:56:16.993Z                Enabling integration modules
 [sentry] DEBUG: Enabling integration argv
[DEBUG] 2021-11-25T18:56:16.993Z                Enabling integration argv
 [sentry] DEBUG: Enabling integration threading
[DEBUG] 2021-11-25T18:56:16.993Z                Enabling integration threading
 [sentry] DEBUG: Enabling integration boto3
[DEBUG] 2021-11-25T18:56:16.993Z                Enabling integration boto3
{'payload': 'hello world!'}
    division_by_zero = 1 / 0line 15, in handler
END RequestId: 83306b75-90b7-4f09-9cae-326af9dfe531
REPORT RequestId: 83306b75-90b7-4f09-9cae-326af9dfe531  Init Duration: 0.34 ms  Duration: 260.11 ms     Billed Duration: 261 ms Memory Size: 3008 MB    Max Memory Used: 3008 MB

Not using the official AWS container as a base

$ docker run -it -p 9000:8080 test
25 Nov 2021 19:35:13,487 [INFO] (rapid) exec '/usr/local/bin/python' (cwd=/, handler=sentry_sdk.integrations.init_serverless_sdk.sentry_lambda_handler)
25 Nov 2021 19:35:15,803 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
25 Nov 2021 19:35:15,803 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory
START RequestId: e6ce0c7c-ec23-4960-b498-195e416c54a0 Version: $LATEST
{'payload': 'hello world!'}
    division_by_zero = 1 / 0n handlerandler()(event, context)dk/__init__.py", line 77, in sentry_lambda_handler
END RequestId: e6ce0c7c-ec23-4960-b498-195e416c54a0
REPORT RequestId: e6ce0c7c-ec23-4960-b498-195e416c54a0  Init Duration: 0.23 ms  Duration: 395.55 ms     Billed Duration: 396 ms Memory Size: 3008 MB    Max Memory Used: 3008 MB

Note

Using the sentry-cli with the DSN works fine.
Installing the sentry-sdk with pip and adding the init to the code also works fine.

The part that doesn't work is the automatic init script.

[Makeshift]

I also tried grabbing the latest zip and setting it up myself, but this also doesn't seem to work:

FROM public.ecr.aws/lambda/python:3.9

RUN yum install -y wget unzip && \
  wget https://github.com/getsentry/sentry-python/releases/download/1.5.0/sentry-python-serverless-1.5.0.zip && \
  unzip sentry-python-serverless-1.5.0.zip && \
  cp -r python/* ${LAMBDA_RUNTIME_DIR} && \
  rm -rf python sentry-python-serverless-1.5.0.zip python

COPY main.py ${LAMBDA_TASK_ROOT}

ENV PYTHONPATH=${LAMBDA_RUNTIME_DIR}:${LAMBDA_TASK_ROOT}
ENV SENTRY_DSN=https://<snip>@sentry.the-iea.tools/5
ENV SENTRY_INITIAL_HANDLER=main.handler
ENV SENTRY_TRACES_SAMPLE_RATE=1.0
ENV SENTRY_DEBUG=1

CMD ["sentry_sdk.integrations.init_serverless_sdk.sentry_lambda_handler"]

def handler(event, context):
  """Entry point for Lambda."""
  print(event)
  division_by_zero = 1 / 0

if __name__ == "__main__":
  handler(None, None)

$ docker run -it -p 9000:8080 test
INFO[0000] exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
INFO[0001] extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
WARN[0001] Cannot list external agents                   error="open /opt/extensions: no such file or directory"
START RequestId: 1e293d77-3f91-45ee-9010-e4ba3f455f31 Version: $LATEST
{'payload': 'hello world!'}
    division_by_zero = 1 / 0line 6, in handlerevent, context)sdk/__init__.py", line 84, in sentry_lambda_handler
END RequestId: 1e293d77-3f91-45ee-9010-e4ba3f455f31
REPORT RequestId: 1e293d77-3f91-45ee-9010-e4ba3f455f31  Init Duration: 0.21 ms  Duration: 466.18 ms     Billed Duration: 467 ms Memory Size: 3008 MB    Max Memory Used: 3008 MB

[AbhiPrasad]

Hey thanks for writing in. We've added this to the backlog to take a look at. In the meantime, PRs and contributions to try to fix this are welcome!

[sarya]

I experienced the same issue and managed to get things working by using the Python 3.8 base image instead of 3.9. Is there any ETA on when more recent versions of Python will be supported?