Could we get a new lightning lora of the 2509 qwen edit model?

[Dervlex]

Please?

[X-niper]

Hi, we are training it and will release it in about 2-7 days.

[JHLEE17]

I currently have some spare GPU resources (at least 16 H100-SXM GPUs) and would be happy to help with your training efforts. If you are interested in experimenting with different hyperparameters, please feel free to reach out, and I can make these resources available to you.

Additionally, I've recently been trying to reproduce the Qwen-Image-Lightning model based on the Self-Forcing-Plus repository and have a few questions. I would appreciate it if you could provide some clarification:

1. I understand that you are using DMD2 for training. For the student and critic models, are you performing full fine-tuning, or LoRA fine-tuning?

1-1. If you are using full fine-tuning, it seems like the memory requirements would be too high to fit the text encoder and all three models on a single GPU. Are you using a combination of Tensor Parallelism (TP) and Fully Sharded Data Parallelism (FSDP) to manage this?

1-2. If you are using LoRA adapters for the student and critic models, do the student, teacher, and critic share a common base model, or does each load an independent base model?

2. The default number of steps for the Qwen-Image model is 50. Is the model trained to progressively reduce the number of steps (e.g., 50 -> 16 -> 8 -> 4), or is it trained to reduce the steps in a single leap (e.g., 50 -> 4 or 50 -> 8)?

3. Could you provide a general overview of the differences in the training processes for V1.0, V1.1, and V2.0?

Thank you for the great works!

[X-niper]

@JHLEE17
HI, thank you so much for your support!

Q.1 Model Architecture and Training Configuration
We designate the three core components as the Generator (G), the Critic (C), and the Teacher (T). For the Qwen-Image model, the Generator is fine-tuned using LoRA with a rank of 64, while the Critic is trained using full-parameter fine-tuning.

Q.1.1 Parallelism and Hardware
We employ Fully Sharded Data Parallelism (FSDP) for the Generator, Critic, Teacher, and the text encoder. Tensor Parallelism was not utilized in this project. For the Wan2.2-Lightning model, we implemented Ulysses sequence parallelism. The training was conducted using either 32 or 64 GPUs, each with 80 GB of memory.

Q.1.2 Model Memory Allocation
The Generator and Teacher models do not share a common base model; consequently, their parameters are stored in separate GPU memory allocations. However, this base-shared approach may be adopted for future distillation of larger-scale models.

Q.2 Distillation Process
The primary distillation was performed in a single stage, directly reducing the sampling steps from 50 to 4 and from 50 to 8. For the 2-step model, a progressive distillation strategy was employed (e.g., 4→2). This 2-step model is not being released, as its performance was found to be inadequate for complex text rendering tasks.

Q.3 Version Differences
The technical distinctions between the three model versions are minimal. Versions 1.0 and 1.1 differ only in their number of training iterations. For Version 2.0, the timestep sampling range was modified from [20, 980] to [10, 1000].