CreateScoreRequest.DatasetRunId not working?

[verdie-g]

I'm using a generated langfuse client from the OpenAPI doc and I'm not able to score a data set run

await langfuseClient.Api.Public.Scores.PostAsync(new CreateScoreRequest
{
    Name = "score-name",
    DatasetRunId = "0465e5fc-7352-4e9c-ba21-4acdc1fffb4a",
    DataType = ScoreDataType.NUMERIC,
    Value = new CreateScoreValue { Double = 123.456 },
    Comment = "score-comment",
});

this run is successful but I can't see the score anywhere. Also, I'm surprised the field is named DatasetRunId and not DatasetRunItemId. I understood that DataSetRun does not exist, it's just a collection of DataSetItem.

Alternatively, I'm using the trace id:

await langfuseClient.Api.Public.Scores.PostAsync(new CreateScoreRequest
{
    Name = "score-name",
    TraceId = "d33a606f1be76f2373bde531b7e97fa6",
    DataType = ScoreDataType.NUMERIC,
    Value = new CreateScoreValue { Double = 123.456 },
    Comment = "score-comment",
});

and this works fine

In the screenshot, I tried to score the first item with DatasetRunId while the second item was scored with TraceId.

[inkeep-ai-assistant[bot]]

In Langfuse, the Score object can be associated with a traceId, observationId, sessionId, or datasetRunId—but only one of these at a time(1)(2)(3). The field datasetRunId is explicitly documented and is used to link a score to an entire dataset run, not to an individual dataset run item(1)(2)(4). By contrast, if you want to score a specific item within a dataset run, you should use the traceId that corresponds to that item(2)(5)(3).

Your observation is correct: there is no documented datasetRunItemId field for the Score object. The workflow is that each dataset run item is represented by a trace, and you should use the trace's ID to score that specific item(5)(3). Scoring with traceId will associate the score with the item (row) in the UI, which matches the behavior you described and what is shown in your screenshot(2)(3).

Scoring with only datasetRunId will associate the score with the entire run, not with a specific item, and this may not display at the row/item level in the UI(2)(3). If you want scores to show up per-row, continue to use traceId as you did in your working example.

📚 Sources:

1. CreateScoreRequest
2. Scores Data Model
3. Custom Scores
4. Score
5. CreateDatasetRunItemRequest

---

Have another question? Just tag @inkeep.

[verdie-g]

@inkeep so there is a DatasetRun object? It doesn't seem like it's documented in the OpenAPI spec, also, it seems to be possible to create a DatasetRunItem without a DatasetRun.

[inkeep-ai-assistant[bot]]

Yes, there is a DatasetRun object in Langfuse. The data model defines a DatasetRun as an object with attributes including id, name, description, metadata, and datasetId, and it represents a run of a dataset through your LLM application(1). The OpenAPI schema also includes a DatasetRunWithItems object, which contains properties such as id, name, datasetId, and an array of datasetRunItems(2).

However, DatasetRun objects are typically created automatically as a side effect when you create a DatasetRunItem and specify a new run name(3). The documented workflow is that creating a DatasetRunItem with a new run name will create (or update) the associated DatasetRun if it does not already exist(3). This means you cannot create a DatasetRun in isolation via OpenAPI; it is always tied to at least one DatasetRunItem(3).

So, while you cannot create a DatasetRun without at least one DatasetRunItem, you also cannot create a DatasetRunItem without a DatasetRun being created or referenced as a side effect—there is always a DatasetRun associated with every DatasetRunItem(1)(2)(3).

📚 Sources:

1. Experiments Data Model
2. DatasetRunWithItems
3. how to create langfuse datasetrun in langfuse-java sdk?

---

Have another question? Just tag @inkeep.

[verdie-g]

@inkeep so what is the purpose of CreateScoreRequest.DatasetRunId in POST /api/public/scores​?

[inkeep-ai-assistant[bot]]

The CreateScoreRequest.DatasetRunId field in POST /api/public/scores is used to associate a score with an entire dataset run, not with individual dataset run items or traces. This allows you to record a single score that represents the overall evaluation or result for a full dataset run. If you want to score specific items (rows) within a dataset run, you should use the traceId field instead, as trace-level scores are linked to individual traces corresponding to dataset run items. DatasetRunId is not intended for per-item scoring—it is for run-level aggregation or summary scores(1), (2).

📚 Sources:

1. 无法直接通过api获取到测试集的平均分数
2. Getting scores efficiently via API for analytics purposes

---

Have another question? Just tag @inkeep.