Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type Error: '<=' not supported between instances of 'LoraConfig' and 'int' #1248

Closed
qy1026 opened this issue Dec 11, 2023 · 1 comment
Closed

Comments

@qy1026
Copy link

qy1026 commented Dec 11, 2023

peft = 0.7.0

When I use the unsloth github repo to faster my qlora finetune code, a TypeError would occur. And the error refers to: https://github.com/huggingface/peft/blob/504d3c832968358498f02b2beac032ed95ef8467/src/peft/tuners/lora/bnb.py#L202C13-L202C92

self.update_layer(adapter_name, r, lora_alpha, lora_dropout, init_lora_weights)

I change the line above into self.update_layer(adapter_name, r.r, lora_alpha, lora_dropout, init_lora_weights)

The TypeError would be fixed.

@BenjaminBossan
Copy link
Member

To me, this sounds like the caller code (from unsloth) is doing something incorrectly. Could you please provide a minimal code sample to reproduce the error?

@qy1026 qy1026 closed this as completed Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants