feat(experiments): Support splitting task JSON response into output and metadata fields in dataset runs

[anuradhapalakurthiTR]

Describe your question

Feature Request

Use Case:
When running experiments via the Experiment Runner SDK, my task function returns a JSON response with multiple fields. I want to separate this response so that:

• One specific field (e.g., message) appears as the Output in the dataset run
• All other fields appear as Metadata in the dataset run

Current Behavior:
The entire JSON response from the task function is stored as the output in dataset runs. There's no documented way to automatically split the response into output vs metadata fields.

Desired Behavior:
Provide a way to specify which fields from a task's JSON response should be treated as output vs metadata, either through:

1. Return value structure (e.g., returning a tuple/dict with separate output and metadata)
2. SDK methods within the task function
3. Configuration in run_experiment() parameters

Example:

def my_task(*, item, **kwargs):
    response = my_application(item.input)
    # response = {"message": "Hello", "tokens": 150, "model": "gpt-4", "latency": 1.2}
    
    # Want: output = "Hello", metadata = {"tokens": 150, "model": "gpt-4", "latency": 1.2}
    return response

### Langfuse Cloud or Self-Hosted?

Langfuse Cloud

### If Self-Hosted

_No response_

### If Langfuse Cloud

_No response_

### SDK and integration versions

_No response_

### Pre-Submission Checklist

- [X] I have checked for existing issues/discussions and consulted Langfuse AI.

[jannikmaierhoefer]

Hi @anuradhapalakurthiTR, thanks!

You can use the Langfuse SDK within your task function to control and manually set the output of your observation or trace. I have created an example here:

from langfuse import get_client
from langfuse.openai import OpenAI
 
# Initialise client
langfuse = get_client()
 
# Define your task function
def my_task(*, item, **kwargs):

  with langfuse.start_as_current_observation(as_type="span", name="my-span") as span:

    question = item["input"]
    response = OpenAI().chat.completions.create(
        model="gpt-4.1", messages=[{"role": "user", "content": question}]
    )

    # Set the output and metadata on the observation level
    span.update(
        output=response.choices[0].message.content,
        metadata=response
    )

    # Set the output and metadata on the observation level
    span.update_trace(
        output=response.choices[0].message.content,
        metadata=response
    )

    return response.choices[0].message.content
 
 
# Run experiment on local data
local_data = [
    {"input": "What is the capital of France?", "expected_output": "Paris"},
    {"input": "What is the capital of Germany?", "expected_output": "Berlin"},
]
 
result = langfuse.run_experiment(
    name="Geography Quiz",
    description="Testing basic functionality",
    data=local_data,
    task=my_task,
)
 
# Use format method to display results
print(result.format())

Let me know if this works for you!

[jannikmaierhoefer]

You can find more on updating observations and traces here: https://langfuse.com/docs/observability/sdk/instrumentation#update-observations