Sentry and serverless sdk not working well together

[zacroker]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.40.5

Steps to Reproduce

Unfortunately this error is sporadic and we are not able to consistently reproduce it, though we can see it is happening from the sentry dashboard.

POTENTIALLY RELATED TO THIS TICKET

1. We have a python django application running in lambda (v3.9)
2. We have sentry configured using sentry-python in our initialisation code
3. We have the python requirements in a lambda layer (serverless-python-requirements)
4. Problem:
   • Occasionally when making API requests we get unexplained 500 errors
   • Occasionally, some of our data pipeline functions crash
5. There is a common set of log lines when these crashes occur:

	2024-04-07T22:09:33.058+10:00	Exception in thread sentry.profiler.ThreadScheduler:
	2024-04-07T22:09:33.058+10:00	Traceback (most recent call last):
	2024-04-07T22:09:33.058+10:00	File "/var/lang/lib/python3.9/threading.py", line 980, in _bootstrap_inner
	2024-04-07T22:09:33.059+10:00	self.run()
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/integrations/threading.py", line 72, in run
	2024-04-07T22:09:33.059+10:00	reraise(*_capture_exception())
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/_compat.py", line 127, in reraise
	2024-04-07T22:09:33.059+10:00	raise value
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/integrations/threading.py", line 70, in run
	2024-04-07T22:09:33.059+10:00	return old_run_func(self, *a, **kw)
	2024-04-07T22:09:33.059+10:00	File "/var/lang/lib/python3.9/threading.py", line 917, in run
	2024-04-07T22:09:33.059+10:00	self._target(*self._args, **self._kwargs)
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/profiler.py", line 943, in run
	2024-04-07T22:09:33.059+10:00	self.sampler()
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/profiler.py", line 828, in _sample_stack
	2024-04-07T22:09:33.059+10:00	sample = [
	2024-04-07T22:09:33.059+10:00	File "/opt/python/sentry_sdk/profiler.py", line 829, in <listcomp>
	2024-04-07T22:09:33.060+10:00	(str(tid), extract_stack(frame, cache, cwd))
	2024-04-07T22:09:33.060+10:00	File "/opt/python/sentry_sdk/profiler.py", line 274, in extract_stack
	2024-04-07T22:09:33.060+10:00	frame_ids = tuple(frame_id(raw_frame) for raw_frame in raw_frames)
	2024-04-07T22:09:33.060+10:00	File "/opt/python/sentry_sdk/profiler.py", line 274, in <genexpr>
	2024-04-07T22:09:33.060+10:00	frame_ids = tuple(frame_id(raw_frame) for raw_frame in raw_frames)
	2024-04-07T22:09:33.060+10:00	File "/opt/python/sentry_sdk/profiler.py", line 301, in frame_id
	2024-04-07T22:09:33.060+10:00	return (raw_frame.f_code.co_filename, raw_frame.f_lineno, get_frame_name(raw_frame))
	2024-04-07T22:09:33.060+10:00	File "/opt/python/sentry_sdk/profiler.py", line 361, in get_frame_name
	2024-04-07T22:09:33.060+10:00	for cls in frame.f_locals["self"].__class__.__mro__:
	2024-04-07T22:09:33.060+10:00	File "/var/task/serverless_sdk/vendor/wrapt/wrappers.py", line 100, in __class__
	2024-04-07T22:09:33.060+10:00	return self.__wrapped__.__class__
	2024-04-07T22:09:33.060+10:00	File "/var/task/serverless_sdk/vendor/wrapt/wrappers.py", line 194, in __getattr__
	2024-04-07T22:09:33.060+10:00	raise ValueError('wrapper has not been initialised')
	2024-04-07T22:09:33.060+10:00	ValueError: wrapper has not been initialised 

Any advice appreciated!

Expected Result

Don't crash 😅

Actual Result

Crash

[sentrivana]

Hey @zacroker, thanks for writing in! 👋🏻

From looking at the logs you posted and some serverless_sdk/wrapt code I don't believe this is related to the other issue where the SDK is unable to start a thread -- in your case the thread is already running.

What you're seeing is a profiler issue. It's trying to inspect something that's not supposed to be inspected and hitting this.

The SDK needs to be smarter here and guard against possible exceptions. We'll fix this. In the meantime, disabling the profiler should get rid of the errors.