Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 69 additions & 4 deletions content/en/llm_observability/instrumentation/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -1230,7 +1230,7 @@
LLMObsSpan taskSpan = LLMObs.startTaskSpan("preprocessDocument", null, "session-141");
... // preprocess document for data extraction
taskSpan.annotateIO(...); // record the input and output
taskSpan.finish();
taskSpan.finish();
}

public String extractData(String document) {
Expand Down Expand Up @@ -1740,7 +1740,7 @@
{{< tabs >}}
{{% tab "Python" %}}

The SDK's `LLMObs.annotate_context()` method returns a context manager that can be used to modify all auto-instrumented spans started while the annotation context is active.
The SDK's `LLMObs.annotation_context()` method returns a context manager that can be used to modify all auto-instrumented spans started while the annotation context is active.

The `LLMObs.annotation_context()` method accepts the following arguments:

Expand All @@ -1756,7 +1756,7 @@

`tags`
: optional - _dictionary_
<br />A dictionary of JSON serializable key-value pairs that users can add as tags on the span. Example keys: `session`, `env`, `system`, and `version`. For more information about tags, see [Getting Started with Tags][1].
<br />A dictionary of JSON serializable key-value pairs that users can add as tags on the span. Example keys: `session`, `env`, `system`, and `version`. For more information about tags, see [Getting Started with Tags](/getting_started/tagging/).

{{% /collapse-content %}}

Expand Down Expand Up @@ -1788,7 +1788,46 @@

{{< /code-block >}}

[1]: /getting_started/tagging/
{{% /tab %}}

{{% tab "Node.js" %}}

The SDK's `llmobs.annotationContext()` accepts a callback function that can be used to modify all auto-instrumented spans started while inside the scope of the callback function.

Check notice on line 1795 in content/en/llm_observability/instrumentation/sdk.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength

Suggestion: Try to keep your sentence length to 25 words or fewer.

The `llmobs.annotationContext()` method accepts the following options on the first argument:

{{% collapse-content title="Options" level="h4" expanded=false id="annotating-autoinstrumented-span-arguments" %}}

`name`
: optional - _str_
<br />Name that overrides the span name for any auto-instrumented spans that are started within the annotation context.

`tags`
: optional - _object_
<br />An object of JSON serializable key-value pairs that users can add as tags on the span. Example keys: `session`, `env`, `system`, and `version`. For more information about tags, see [Getting Started with Tags](/getting_started/tagging/).

{{% /collapse-content %}}

#### Example

{{< code-block lang="javascript" >}}
const { llmobs } = require('dd-trace');

function ragWorkflow(userQuestion) {
const contextStr = retrieveDocuments(userQuestion).join(" ");

const completion = await llmobs.annotationContext({
tags: {
retrieval_strategy: "semantic_similarity"
},
name: "augmented_generation"
}, async () => {
const completion = await openai_client.chat.completions.create(...);
return completion.choices[0].message.content;
});
}

{{< /code-block >}}

{{% /tab %}}
{{< /tabs >}}
Expand Down Expand Up @@ -2161,6 +2200,32 @@
llmobs.registerProcessor(redactProcessor)
{{< /code-block >}}

### Example: conditional modification with auto-instrumentation

When using auto instrumentation, the span is not always contextually accessible. To conditionally modify the inputs and outputs on auto-instrumented spans, `llmobs.annotationContext()` can be used in addition to a span processor.

{{< code-block lang="javascript" >}}
const { llmobs } = require('dd-trace');

function redactProcessor(span) {
if (span.getTag("no_input") == "true") {
for (const message of span.input) {
message.content = "";
}
}

return span;
}

llmobs.registerProcessor(redactProcessor);

async function callOpenai() {
await llmobs.annotationContext({ tags: { no_input: "true" } }, async () => {
// make call to openai
});
}
{{< /code-block >}}

### Example: preventing spans from being emitted

{{< code-block lang="javascript" >}}
Expand Down
Loading