How to specify the split (train/validation) for the dataset in cli

[ma7555]

How can I specify the split to use for training and validation?

CUDA_VISIBLE_DEVICES=0 MAX_PIXELS=262144 \
swift sft \
    --model LLM-Research/gemma-3-1b-it \
    --train_type full \
    --dataset 'swift/path-vqa#train' \
    --val_dataset 'swift/path-vqa#validation' \
    --torch_dtype bfloat16 \
    --num_train_epochs 3 \

Of course this will fail as #train is treated as a subset not a split. How can I specify the split?

[Jintao-Huang]

https://github.com/modelscope/ms-swift/blob/main/swift/llm/dataset/dataset/mllm.py#L174

use 'modelscope/coco_2014_caption:validation'

[ma7555]

Hello @Jintao-Huang, I have to use swift/path-vqa it is not optional.

[Jintao-Huang]

https://github.com/modelscope/ms-swift/blob/main/swift/llm/dataset/data/dataset_info.json#L612

You may need to modify the source code to resolve the issue; perhaps the following modification:

https://github.com/modelscope/ms-swift/blob/main/swift/llm/dataset/data/dataset_info.json#L105

    {
        "ms_dataset_id": "swift/path-vqa",
        "hf_dataset_id": "flaviagiammarino/path-vqa",
        "subsets": [{
            "name": "train",
            "split": ["train"]
        },
        {
            "name": "validation",
            "split": ["validation"]
        }]
        "columns": {
            "question": "query",
            "answer": "response"
        },
        "tags": ["multi-modal", "vqa", "medical"]
    },

[github-actions]

This issue has been inactive for over 3 months and will be automatically closed in 7 days. If this issue is still relevant, please reply to this message.

[github-actions]

This issue has been automatically closed due to inactivity. If needed, it can be reopened.