Add fsdp+qlora support

[deep-diver]

This PR is to add FSDP+QLoRA support with the following changes:

• add recipes/accelerate_configs/fsdp+qlora.yaml
• update versions of peft>=0.9.0 and bitsandbytes>=0.43.0 dependencies
• bnb_4bit_quant_storage field in ModelArguments
• set bnb_4bit_quant_storage of BitsAndBytesConfig

With these changes, I have confirmed FSDP+QLoRA works within my local setup (2 x A6000).

[lewtun]

Thanks a lot for the nice addition @deep-diver ! Would you like to add a small example on how to run FSDP + QLoRA here? https://github.com/huggingface/alignment-handbook/tree/main/scripts#fine-tuning

Let's also rename the accelerate config and then we can merge

[lewtun]

Let's call this just fsdp.yaml since I think it will also work with full training

[deep-diver]

For FSDP+QLoRA, We need to set fsdp_cpu_ram_efficient_loading=true, fsdp_use_orig_params=false and fsdp_offload_params=true(cpu offloading) when using Accelerate config according to this doc.

Since this is a recipe, I think it would be handy to provide separate yaml who wants to use FSDP+QLoRA out of the box. WDYT?

[lewtun]

Sounds great, can we then rename the file to fsdp_qlora.yaml please?

[deep-diver]

@lewtun

done! :)

[deep-diver]

@lewtun

Besides keeping or removing fsdp+qlora.yaml discussion, I made additional commit for adding an example on https://github.com/huggingface/alignment-handbook/tree/main/scripts#fine-tuning. Please take a look!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[deep-diver]

@lewtun

reminder

[deep-diver]

@lewtun

reminder. I addressed your comments :)