fix(openai): Don't let instrumentation raise into user code

[FromCSUZhou]

Fixes #7222

What

The OpenAI integration can raise out of the instrumented create() call, so an SDK problem surfaces as an application error and replaces whatever the API actually returned:

  File "sentry_sdk/integrations/openai.py", line 1532, in _new_sync_responses_create
    _set_responses_api_output_data(
  File "sentry_sdk/integrations/openai.py", line 746, in _set_common_output_data
    for output in response.output:
                  ^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

Why

Two independent causes, both addressed here.

1. response.output is never checked for None. response.choices got an is not None check after #5071, but the Responses API branch still does a bare hasattr(response, "output") in _set_common_output_data and _calculate_responses_token_usage.

2. The non-streaming paths lost their capture_internal_exceptions() wrapper. Before #4612 the whole response-handling block ran inside with capture_internal_exceptions():. Splitting it into _set_*_input_data / _set_*_output_data moved that code out from under the wrapper, so errors there now propagate into user code. The streaming iterators kept their protection; the non-streaming paths did not — which is why <2.34 is unaffected.

(1) fixes the crash that was actually hit; (2) makes this class of bug degrade to a missing span attribute rather than an application crash, per the SDK contract in CONTRIBUTING.md ("Users do not expect their application to crash").

How it is reached in practice

Through an OpenAI-compatible gateway. When a gateway has already flushed 200 OK — for example after sending keep-alive padding while it buffers a large request — and only then learns the upstream call failed, it cannot change the status code, so it reports the failure in the body:

{"error": {"message": "... At least one of the image dimensions exceed max allowed size: 8000 pixels", "code": 400}}

The openai client parses this into a model object with output / choices unset. Instrumenting it raises, and the application sees TypeError: 'NoneType' object is not iterable instead of the real message — so retry and fallback logic keyed on the upstream error stops working.

Minimal reproduction, no network needed:

from unittest import mock

import sentry_sdk
from openai import OpenAI
from openai.types.responses import Response
from sentry_sdk.integrations.openai import OpenAIIntegration

sentry_sdk.init(
    dsn="https://public@o1.ingest.sentry.io/1",
    traces_sample_rate=1.0,
    integrations=[OpenAIIntegration()],
    transport=lambda event: None,
)

gateway_error = Response.construct(
    error={"message": "upstream failed", "code": 400},
    output=None,
)

client = OpenAI(api_key="z")
client.responses._post = mock.Mock(return_value=gateway_error)

with sentry_sdk.start_transaction(name="tx"):
    response = client.responses.create(model="gpt-4o", input="hello")
    print(response.error)  # raises TypeError before this line on master

Changes

• _calculate_responses_token_usage and _set_common_output_data: require response.output is not None, mirroring the existing response.choices is not None check. A response with output=None now falls through to the same branch a response with choices=None already does.
• _set_responses_api_input_data, _set_completions_api_input_data, _set_embeddings_input_data and _set_common_output_data are now thin wrappers that run the recording work inside capture_internal_exceptions(). The bodies moved unchanged into _record_* helpers.
• _set_common_output_data finishes the span after the guarded block rather than at the end of each branch, so a failure while recording cannot leave an unfinished span behind.

Regarding #4853 (asyncio.CancelledError swallowed by capture_internal_exceptions()): that report concerned a wrapper spanning the user's iterator consumption. The blocks wrapped here contain only synchronous instrumentation code with no await points and no calls back into user code, so a task cancellation cannot be absorbed by them.

Testing

Added to tests/integrations/openai/test_openai.py:

• test_responses_api_none_output_does_not_crash / ..._async — a Responses object with output=None is returned to the caller, and the span is still recorded and finished.
• test_chat_completion_none_choices_does_not_crash — regression guard for the choices=None path fixed in fix(openai): Check response text is present to avoid AttributeError #5081.
• test_instrumentation_output_error_does_not_propagate / test_instrumentation_input_error_does_not_propagate — an unexpected error while recording request or response data does not reach the caller, and the span is still finished.

4 of the 5 fail on master and pass with this change; the fifth guards the already-fixed choices=None path.

$ pytest tests/integrations/openai -q
699 passed

(694 before this change, 5 new.) ruff format --check and ruff check are clean on both touched files, and mypy reports no new errors relative to master.