Support dataset splits (train/dev/test) in the Experiment Runner SDK

[jannikmaierhoefer]

Describe the feature or potential improvement

Users might structure their evaluation datasets with explicit splits (e.g., train, dev, test) and store this split information in the item metadata (e.g., "split": "test").

However, the current run_experiment SDK method does not allow filtering dataset items by split, which makes it difficult to run experiments only on a subset of the dataset.

Feature request:

Add first-class support for dataset splits to the Experiment Runner SDK. This could include:
Allowing run_experiment to accept a filter (e.g., split="test" or a metadata query).

Additional information

No response