Add extract adapters support for dora and loha

[xiaoyu-work]

Describe your changes

Add extract adapters support for dora and loha.

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
• Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

(Optional) Issue link

[jambayk]

Can you remove this condition in the peft export helper

Olive/olive/common/hf/peft.py

Line 48 in 50f360a

or getattr(module, "use_dora", {}).get(module.active_adapters[0], False)

?

Otherwise, the lora scaling factors appears in the graph and would need to be extracted also since it is not the same for all adapters.

[xiaoyu-work]

Removed.

[jambayk]

I looked at the exported graph for a dora model after the scaling change above. the lora matmuls appear twice in the computation.
Also, the Add weight doesn't need to be extracted. It is just the transpose of the base weight and comes from this step https://github.com/huggingface/peft/blob/363c14e673a12d19f951609d06221962d5c3eb2a/src/peft/tuners/lora/dora.py#L78. This is constant for a model.

What needs to be extracted is the weight in the Div node that depends on the lora weights.
https://github.com/huggingface/peft/blob/363c14e673a12d19f951609d06221962d5c3eb2a/src/peft/tuners/lora/dora.py#L86
https://github.com/huggingface/peft/blob/363c14e673a12d19f951609d06221962d5c3eb2a/src/peft/tuners/lora/dora.py#L63

[jambayk]

Ideally, during export, we can precompute https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/dora.py#L70-L86 so that ConstantOfShape -> Reshape doesn't appear in the graph (more compute + memory because of the duplicated transposed base weight) and we just have the mag_norm_scale weight directly in the graph. But I think this can be done later if needed and is out of scope for this PR.

[xiaoyu-work]

Thanks for pointing this out! I double checked the graph and updated the code to use Div rather Add for now!