Convert alphas for embedders for sd-scripts to ai toolkit conversion

[rockerBOO]

What does this PR do?

Converts the alpha + weight for embedders (guidance, txt/img, vector, timestep) from Kohya sd-scripts into diffusers format. There was a # TODO: alphas and this uses the function to convert these over using the one with alphas. Remaining is the final_layer but that is a bit different due to the transform function.

Added a test from a LoRA I made with these using Kohya sd-scripts.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul

[sayakpaul]

Why is this going away? From what I understand, assign_remaining_weights() is doing the same thing as what the changes in this PR are doing no?

[rockerBOO]

It isn't converting the alphas so instead of making assign_remaining_weights do the alpha I thought to use the other function (_convert_to_ai_toolkit) here. Maybe I'm missing the key reason this was being used instead.

[sayakpaul]

Ah okay. That makes sense.

Could we run the integration tests for Flux LoRA before we merge this PR?

diffusers/tests/lora/test_lora_layers_flux.py

Line 816 in 751e250

class FluxLoRAIntegrationTests(unittest.TestCase):

[rockerBOO]

I couldn't run all the tests due to my PC and even these I had to disable all the skips for this to run. But I was able to complete the test for the new test. If you need more of the tests run I can try though.

RUN_SLOW=1 RUN_NIGHTLY=1 python -m pytest tests/lora/test_lora_layers_flux.py -k test_flux_kohya_embedders_conversion
platform linux -- Python 3.13.5, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/rockerboo/code/others/diffusers
configfile: pyproject.toml
plugins: timeout-2.4.0, requests-mock-1.10.0, xdist-3.8.0
collected 122 items / 121 deselected / 1 selected

tests/lora/test_lora_layers_flux.py .

1 passed, 121 deselected, 3 warnings in 555.38s (0:09:15)

[sayakpaul]

Sorry, I meant the other tests that we have in the test suite. Let me try to run them.

[sayakpaul]

@bot /style

[github-actions]

Style bot fixed some files and pushed the changes.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Just ran all the tests and they pass. Will merge the PR after the CI is through.

[sayakpaul]

Super cool!