[Bug] Full Finetune: Tensors of floating point dtype can require gradients

[charliedream1]

1. Did you update? pip install --upgrade unsloth unsloth_zoo
2. Colab or Kaggle or local / cloud
3. Number GPUs used, use nvidia-smi
4. Which notebook?
5. Paste Unsloth printout with 🦥 sloth emoji
6. Which trainer? SFTTrainer, GRPOTrainer etc
7. Minimal code to reproduce error Remove Hugging Face token!

For quick replies, got to https://discord.com/invite/unsloth.
Have you tried https://docs.unsloth.ai/basics/errors-troubleshooting

For full finetune, it gives out: RuntimeError: only Tensors of floating point dtype can require gradients

Model used: unsloth/Qwen3-0.6B-Base (I manually downloaded from the website)

Model loaded as below:

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = mdl_path, load_in_4bit = False,
    max_seq_length = max_seq_length,
    dtype = None,
    full_finetuning = True, # We have full finetuning now!
)

[mmathew23]

Hi could you provide more details or a minimal script that would reproduce the error? Going to be hard to help based of what you're showing. Do you see the same error running via one of our notebooks on colab?

[Sneakr]

1. Did you update? pip install --upgrade unsloth unsloth_zoo
2. Colab or Kaggle or local / cloud
3. Number GPUs used, use nvidia-smi
4. Which notebook?
5. Paste Unsloth printout with 🦥 sloth emoji
6. Which trainer? SFTTrainer, GRPOTrainer etc
7. Minimal code to reproduce error Remove Hugging Face token!

For quick replies, got to https://discord.com/invite/unsloth. Have you tried https://docs.unsloth.ai/basics/errors-troubleshooting

For full finetune, it gives out: RuntimeError: only Tensors of floating point dtype can require gradients

Model used: unsloth/Qwen3-0.6B-Base (I manually downloaded from the website)

Model loaded as below:

model, tokenizer = FastLanguageModel.from_pretrained(
model_name = mdl_path, load_in_4bit = False,
max_seq_length = max_seq_length,
dtype = None,
full_finetuning = True, # We have full finetuning now!
)

Try:
dtype = torch.bfloat16

[charliedream1]

I'm using torch 2.4+cuda12.1, xformers==0.0.27.post2, unsloth==2025.4.7, transformers==4.51.3,bitsandbytes==0.45.5

for models, I tried unsloth/Qwen3-0.6B-Base, Qwen3-0.6B-Base-bnb-4bit, I downloaded them manually

I tried to set as 4 bit, 8 bit, bf16, fp32.

Here is my code, problem still happen:

import os
# needed as this function doesn't like it when the lm_head has its size changed
from unsloth import tokenizer_utils

import torch
major_version, minor_version = torch.cuda.get_device_capability()
print(f"Major: {major_version}, Minor: {minor_version}")
from datasets import load_dataset
import datasets
from trl import SFTTrainer
import pandas as pd
import numpy as np
import os
import pandas as pd
import numpy as np
from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments, Trainer
from typing import Tuple
import warnings
from typing import Any, Dict, List, Union
from transformers import DataCollatorForLanguageModeling
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

def make_dirs(*dirs):
    for i in dirs:
        if not os.path.exists(i):
            os.makedirs(i)

# 1. Initialization
work_path = '/data/workspace/unsloth'
train_data_path = os.path.join(work_path, 'train.parquet')
valid_data_path = os.path.join(work_path, 'validation.parquet')
# NOTICE: 4 bit model can not be used
mdl_path = '/mdl/mdl_zoo/unsloth/Qwen3-0.6B-Base'

save_dir = '/mdl/out/unsloth'
make_dirs(save_dir)
save_path = os.path.join(save_dir, 'Qwen3-0.6B-base-full')
data_random_sample = -1  # -1 means no random sample
# ===

NUM_CLASSES = 3 # number of classes in the csv
max_seq_length = 12768 # Choose any! We auto support RoPE Scaling internally!
dtype = torch.bfloat16 # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = False

# =============================
def do_nothing(*args, **kwargs):
    pass

tokenizer_utils.fix_untrained_tokens = do_nothing

train_dataset = load_dataset("parquet", data_files=train_data_path)['train']
if data_random_sample > 0:
    train_dataset = train_dataset.shuffle(seed=3407).select(range(data_random_sample))
    print(f"Random sample {data_random_sample} from {len(train_dataset)}")

# 0. data format
prompt = """Here is a financial news:
{}

Classify this news into one of the following:
class 1: Bullish
class 2: Neutral
class 3: Bearish

SOLUTION
The correct answer is: class {}"""

def formatting_prompts_func(dataset_):
    # this is to fix an issue with the transformers library where the first time this function is called, it is called with a string for some reason
    if isinstance(dataset_['text'], str):
        return [" "]*100

    texts = []
    for i in range(len(dataset_['text'])):
        text_ = dataset_['text'][i]
        label_ = dataset_['label'][i] # the csv is setup so that the label column corresponds exactly to the 3 classes defined above in the prompt (important)

        text = prompt.format(text_, label_)

        texts.append(text)
    return texts


# 1. loading model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = mdl_path, load_in_4bit = load_in_4bit,
    max_seq_length = max_seq_length,
    dtype = dtype,
    full_finetuning = True, # We have full finetuning now!
)

# 2. change output head to adapt to new class number
number_token_ids = []
for i in range(0, NUM_CLASSES+1):
    number_token_ids.append(tokenizer.encode(str(i), add_special_tokens=False)[0])
# keep only the number tokens from lm_head
par = torch.nn.Parameter(model.lm_head.weight[number_token_ids, :])

old_shape = model.lm_head.weight.shape
old_size = old_shape[0]
print(par.shape)
print(old_shape)

model.lm_head.weight = par

reverse_map = {value: idx for idx, value in enumerate(number_token_ids)} # will be used later to convert an idx from the old tokenizer to the new lm_head
print('reverse_map:', reverse_map)

print("trainable parameters:", sum(p.numel() for p in model.parameters() if p.requires_grad))

# 3. Define Data loading
# this custom collator makes it so the model trains only on the last token of the sequence. It also maps from the old tokenizer to the new lm_head indices
class DataCollatorForLastTokenLM(DataCollatorForLanguageModeling):
    def __init__(
        self,
        *args,
        mlm: bool = False,
        ignore_index: int = -100,
        **kwargs,
    ):
        super().__init__(*args, mlm=mlm, **kwargs)
        self.ignore_index = ignore_index

    def torch_call(self, examples: List[Union[List[int], Any, Dict[str, Any]]]) -> Dict[str, Any]:
        batch = super().torch_call(examples)

        for i in range(len(examples)):
            # Find the last non-padding token
            last_token_idx = (batch["labels"][i] != self.ignore_index).nonzero()[-1].item()
            # Set all labels to ignore_index except for the last token
            batch["labels"][i, :last_token_idx] = self.ignore_index
            # If the last token in the text is, for example, "2", then this was processed with the old tokenizer into number_token_ids[2]
            # But we don't actually want this because number_token_ids[2] could be something like 27, which is now undefined in the new lm_head. So we map it to the new lm_head index.
            batch["labels"][i, last_token_idx] = reverse_map[ batch["labels"][i, last_token_idx].item() ]

        return batch

collator = DataCollatorForLastTokenLM(tokenizer=tokenizer)

# 4. Start to train
trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = train_dataset,
    max_seq_length = max_seq_length,
    dataset_num_proc = 1,
    packing = False, # not needed because group_by_length is True
    args = TrainingArguments(
        per_device_train_batch_size = 8,
        gradient_accumulation_steps = 1,
        warmup_steps = 10,
        learning_rate = 1e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "cosine",
        seed = 3407,
        output_dir = "outputs",
        num_train_epochs = 1,  # Set this for 1 full training run.
        # max_steps = 5000,
        # report_to = "wandb",  # tensorboard， none
        report_to = "none",
        group_by_length = True,
    ),
    formatting_func=formatting_prompts_func,
    data_collator=collator,
)


# @title Show current memory stats
gpu_stats = torch.cuda.get_device_properties(0)
start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)
print(f"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.")
print(f"{start_gpu_memory} GB of memory reserved.")

trainer_stats = trainer.train()

# @title Show final memory and time stats
used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
used_memory_for_lora = round(used_memory - start_gpu_memory, 3)
used_percentage = round(used_memory / max_memory * 100, 3)
lora_percentage = round(used_memory_for_lora / max_memory * 100, 3)
print(f"{trainer_stats.metrics['train_runtime']} seconds used for training.")
print(
    f"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training."
)
print(f"Peak reserved memory = {used_memory} GB.")
print(f"Peak reserved memory for training = {used_memory_for_lora} GB.")
print(f"Peak reserved memory % of max memory = {used_percentage} %.")
print(f"Peak reserved memory for training % of max memory = {lora_percentage} %.")

# 5. save lora model
# Save the current (trimmed) lm_head
trimmed_lm_head = model.lm_head.weight.data.clone()

# Create a new lm_head with shape [old_size, hidden_dim]
hidden_dim = trimmed_lm_head.shape[1]
# remake the old lm_head but with unused tokens having -100 weights (improves compatibility with libraries like vllm)
new_lm_head = torch.full((old_size, hidden_dim), -100.0, dtype=trimmed_lm_head.dtype, device=trimmed_lm_head.device)

# Fill in the weights for the allowed tokens (number_token_ids)
for new_idx, orig_token_id in enumerate(number_token_ids):
    new_lm_head[orig_token_id] = trimmed_lm_head[new_idx]

# Update the model's lm_head weight
with torch.no_grad():
    # Create a new linear layer with the same input dimension but full vocab size
    new_lm_head_module = torch.nn.Linear(hidden_dim, old_size, bias=False, device=model.device)
    # Set its weights
    new_lm_head_module.weight.data.copy_(new_lm_head)
    # Replace the module
    model.lm_head.modules_to_save["default"] = new_lm_head_module

print(f"Remade lm_head: shape = {model.lm_head.weight.shape}. Allowed tokens: {number_token_ids}")

print(f'save lora to: {save_path}')
model.save_pretrained(save_path)

[linh222]

I had the same error when loading the model from downloaded files. I tried to load the model from Huggingface ID, and it worked. It requires a model with float16 for full fine-tuning.

[github-actions]

Is this issue still important to you? Apologies in advance we might have missed this issue as well. For faster response times, please post on our Reddit server - https://www.reddit.com/r/unsloth or our Discord - https://discord.com/invite/unsloth