ddtrace-run error using OpenAI Chat Completions API

[ykeremy]

Summary of problem

Can't use the chat completions API using OpenAI python SDK. I tried with these models: gpt-4-1106-preview, gpt-4-vision-preview (with/without image), gpt-4.

I also tried setting these env variables but it doesn't get rid of the problem:

DD_OPENAI_LOGS_ENABLED=false
DD_OPENAI_METRICS_ENABLED=false

Which version of dd-trace-py are you using?

2.3.1

Which version of pip are you using?

Tried both Poetry (version 1.6.1) and pip 23.3.1

Which libraries and their versions are you using?

openai version 1.3.5

How can we reproduce your problem?

The below snippet works without ddtrace-run but gives the error shared below when I run it with ddtrace-run.

from openai import OpenAI

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Hello, I believe I'm a bug!",
            },
        ]
    }
]

# Set your API key
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=messages,
)
print(response.choices[0].message.content)

What is the result that you get?

    response = await available_client.client.chat.completions.with_raw_response.create(**chat_completion_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openai/_response.py", line 261, in wrapped
    return cast(APIResponse[R], await func(*args, **kwargs))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/patch.py", line 433, in patched_endpoint
    g.send(None)
  File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/patch.py", line 365, in _traced_endpoint
    hook.send(None)
  File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 69, in handle_request
    self._record_request(pin, integration, span, args, kwargs)
  File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 254, in _record_request
    content = integration.trunc(m.get("content", ""))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/_trace_utils_llm.py", line 138, in trunc
    text = text.replace("\n", "\\n").replace("\t", "\\t")
           ^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'replace'

What is the result that you expected?

No errors

[Yun-Kim]

Hi @ykeremy, thanks for reaching out!

Currently we don't yet support image inputs for the chat completions endpoint. We'll work on a fix in the meantime.

[bishoco]

I am having a similar issue using ddtrace and openai. I'm using ddtrace 2.3.1 and openai 1.3.5. If this is a separate issue, I can open up a new issue. I had to disable DataDog.

Here is the stack trace of the error:
self.llm_client.chat.completions.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/_utils/_utils.py", line 272, in wrapper def wrapper(*args: object, **kwargs: object) -> object: File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/patch.py", line 401, in patched_endpoint g.send(None) File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/patch.py", line 365, in _traced_endpoint hook.send(None) File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 69, in handle_request self._record_request(pin, integration, span, args, kwargs) File "/usr/local/lib/python3.11/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 254, in _record_request content = integration.trunc(m.get("content", "")) ^^^^^ File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 755, in getattr raise AttributeError(f'{type(self).name!r} object has no attribute {item!r}') from exc AttributeError: 'ChatCompletionMessage' object has no attribute 'get'

[Yun-Kim]

Hi @bishoco, could you also include the input that you're sending to the chat completions endpoint? This would help me reproduce your error. Thanks!

[bishoco]

@Yun-Kim This is how I'm calling it:

model="gpt-4-1106-preview"
messages = [{"role": "user", "content": "Hello!"}]

stream = self.llm_client.chat.completions.create( model=model, messages=messages, stream=True )

[Yun-Kim]

@bishoco I'm not able to reproduce your error, using openai==1.3.5 and ddtrace==2.3.1 with this setup:

import openai
client = openai.OpenAI()
model = "gpt-4-1106-preview"
messages = [{"role": "user", "content": "Hello!"}]
stream = client.chat.completions.create(model=model, messages=messages, stream=True)

Do you see anything different about this than how you're calling the chat completions endpoint?

[bishoco]

@Yun-Kim I just gave you a simplified version of the messages I'm sending. The messages I'm sending are actually a bit different. Let me see if I can pull what I'm actually sending in messages to see if that reproduces the error. I'll get back to you.

[Yun-Kim]

Hi @bishoco, please create a new issue if you're still seeing the error, it is most likely separate from this issue as we've fixed this in #7759.