Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Conversation

@Satrat
Copy link

@Satrat Satrat commented Mar 8, 2024

Asana ticket: https://app.asana.com/0/1206109050183159/1206788571042422/f

We had missed support for passing in an instantiated teacher to the finetuning code, this PR adds in it

Testing

script would fail on main, training starts as intended here:

from sparseml.transformers import compress, SparseAutoModelForCausalLM, SparseAutoTokenizer, load_dataset

model = SparseAutoModelForCausalLM.from_pretrained("zoo:llama2-7b-ultrachat200k_llama2_pretrain-base", device_map="auto")
teacher = SparseAutoModelForCausalLM.from_pretrained("zoo:llama2-7b-open_platypus_orca_llama2_pretrain-base", device_map="auto")
tokenizer = SparseAutoTokenizer.from_pretrained("zoo:llama2-7b-ultrachat200k_llama2_pretrain-base")
recipe = "zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned40"
dataset = load_dataset("garage-bAInd/Open-Platypus")

def format_data(data):
    data["text"] = data["instruction"] + data["output"]
    return data

dataset = dataset.map(format_data)

compress(
    model=model,
    tokenizer=tokenizer,
    distill_teacher=teacher,
    dataset=dataset,
    recipe=recipe,
    num_train_epochs=1,
    output_dir="./output",
)

@Satrat Satrat requested review from bfineran, dbogunowicz, horheynm and rahul-tuli and removed request for dbogunowicz and horheynm March 8, 2024 19:41
@Satrat Satrat merged commit 2bd0bd8 into main Mar 8, 2024
@Satrat Satrat deleted the fix_teacher_py branch March 8, 2024 21:50
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants