Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new column for sft fine tuning with prepare_dataset #309

Merged
merged 1 commit into from
Jan 31, 2024

Conversation

plaguss
Copy link
Contributor

@plaguss plaguss commented Jan 30, 2024

Description

This PR adds functionality on the prepare_dataset function to generate a new column to be used for SFT:

from distilabel.utils import prepare_dataset

dataset = load_dataset("argilla/distilabel-intel-orca-dpo-pairs", split="train")
dataset.task = JudgeLMTask()
dataset_binarized_random = prepare_dataset(dataset, strategy="random", keep_ties=True, sft=True). # 'sft' defaults to False

This dataset will include a new column messages based on the content of the chosen response.
Initially it's not well documented as it just offers a simple way to extend the functionality just on the preference datasets using the same function, but this could be more general. Once refactored, we can use a special section for the functionality.

Example of use of this column from the alignment handbook

Closes #276.

@plaguss plaguss self-assigned this Jan 31, 2024
@plaguss plaguss added this to the 0.5.0 milestone Jan 31, 2024
@plaguss plaguss merged commit 5bd84bc into main Jan 31, 2024
4 checks passed
@plaguss plaguss deleted the feat/prepare-dataset-sft branch January 31, 2024 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Add messages to the new prepare_dataset function to allow for sft fine tuning
1 participant