Don't swallow userland exceptions in openai #4861

sl0thentr0py · 2025-09-24T15:03:10Z

Issues

resolves: asyncio.CancelledError exceptions are swallowed by capture_internal_exceptions() #4853
resolves: PY-1865

linear · 2025-09-24T15:03:14Z

PY-1865 asyncio.CancelledError exceptions are swallowed by capture_internal_exceptions()

codecov · 2025-09-24T15:07:19Z

❌ 24 Tests Failed:

Tests completed	Failed	Passed	Skipped
23515	24	23491	2304

View the top 3 failed test(s) by shortest run time

tests.integrations.aws_lambda.test_aws_lambda::test_basic_exception

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_error_has_existing_trace_context[RaiseErrorPerformanceEnabled]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_error_has_new_trace_context[RaiseErrorPerformanceDisabled]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_error_has_new_trace_context[RaiseErrorPerformanceEnabled]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_headers[empty dict headers]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_headers[empty list headers]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_headers[empty string headers]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_headers[no headers]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_headers[none headers]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_init_error

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as bool]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as dict]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as empty list]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as float]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as integer]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as list of dicts]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_non_dict_event[event as string]

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_request_data

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_span_origin

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_timeout_error

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_trace_continuation

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_traces_sampler_has_correct_sampling_context

Stack Traces | 0s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_error_has_existing_trace_context[RaiseErrorPerformanceDisabled]

Stack Traces | 0.001s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

tests.integrations.aws_lambda.test_aws_lambda::test_basic_no_exception

Stack Traces | 71.5s run time

.../integrations/aws_lambda/test_aws_lambda.py:70: in test_environment
    LocalLambdaStack.wait_for_stack()
.../integrations/aws_lambda/utils.py:221: in wait_for_stack
    raise TimeoutError(
E   TimeoutError: AWS SAM failed to start within 60 seconds. (Maybe Docker is not running?)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

alexander-alderman-webb

Looks good to me!

Is it worth checking the other AI integrations and adding tests? From a quick search, the same issue could affect the cohere integration. Not blocking for this PR.

sl0thentr0py · 2025-09-24T15:34:39Z

I usually prefer not to add negative tests such as 'test something that shouldn't have happened didn't happen' because they add cruft over time but that's just personal preference. We're simply fixing a badly written integration here and there's a linked github bug report so that's good enough for documentation.

As for cohere, feel free to make a fix!

sl0thentr0py requested a review from a team as a code owner September 24, 2025 15:03

This comment was marked as outdated.

Sign in to view

Don't swallow userland exceptions in openai

6d0096c

sl0thentr0py force-pushed the neel/fix-openai-swallowed-exc branch from 96e565e to 6d0096c Compare September 24, 2025 15:12

alexander-alderman-webb approved these changes Sep 24, 2025

View reviewed changes

sl0thentr0py merged commit 6325924 into master Sep 24, 2025
113 checks passed

sl0thentr0py deleted the neel/fix-openai-swallowed-exc branch September 24, 2025 15:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Don't swallow userland exceptions in openai #4861

Don't swallow userland exceptions in openai #4861

Uh oh!

sl0thentr0py commented Sep 24, 2025

Uh oh!

linear bot commented Sep 24, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Sep 24, 2025 •

edited

Loading

Uh oh!

alexander-alderman-webb left a comment

Uh oh!

sl0thentr0py commented Sep 24, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Don't swallow userland exceptions in openai #4861

Don't swallow userland exceptions in openai #4861

Uh oh!

Conversation

sl0thentr0py commented Sep 24, 2025

Issues

Uh oh!

linear bot commented Sep 24, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 24 Tests Failed:

Uh oh!

alexander-alderman-webb left a comment

Choose a reason for hiding this comment

Uh oh!

sl0thentr0py commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Sep 24, 2025 •

edited

Loading

sl0thentr0py commented Sep 24, 2025 •

edited

Loading