Added usable LoRA fine-tuning (experimental, tested only on Acoustic models)

ver0.0.1: Added usable LoRA fine-tuning (experimental, tested only on Acoustic models)

This is the first version of our DSRX. Based on the openvpi fork (November 2024 version), we have introduced an experimental LoRA fine-tuning mechanism.

LoRA fine-tuning employs a low-rank decomposition matrix placed alongside the main model. By keeping the pre-trained model parameters frozen while introducing trainable low-rank matrices, it simulates the effects of full-parameter fine-tuning, significantly reducing training resource consumption and duration. In our tests, using only a 1.5-minute Japanese voice dataset (from "Blue Archive——Yutori Natsu"), with a Chinese-Japanese bilingual model (parameters: 160k steps, 512*20 network size, WaveNet backbone, 64 batch size) as the base model, and LoRA parameters (8 rank, 16 alpha, injected into 'fs2' & 'diffusion' modules, training bias enabled), we achieved excellent singing synthesis and cross-lingual performance after 500 training steps.

To use LoRA fine-tuning, please utilize configs/acoustic-lora.yaml for preprocessing and training. Acoustic model inference and ONNX export have been tested and can be properly integrated with OpenUtau.

Note: We have not yet conducted experiments with both LoRA fine-tuning and forgetting/frozen layer fine-tuning enabled simultaneously. However, we currently advise against enabling both concurrently.

If you encounter bugs during use, please submit a PR or issue to this repo. We will evaluate and improve them as soon as possible.

Kouon Project Team
September 13, 2025, late at night

Appendix: Brief explanation of new parameters in acoustic-yaml (based on openvpi fork, November 2024 version):

binarization_args.allow_missing_phonemes: This parameter accepts a boolean value, defaulting to True in LoRA preprocessing and training. When enabled, preprocessing skips phonemes missing from the dictionary, prints missing phoneme warnings during preprocessing, and creates missing_phonemes.json in the preprocessing folder. This option is strongly recommended only for forgetting/frozen layer or LoRA fine-tuning.

lora:
a. lora.enabled: Accepts a boolean value, defaulting to True, indicating that LoRA fine-tuning is enabled.
b. lora.rank: Accepts an integer value as the low-rank parameter, controlling adapter capacity and GPU memory consumption, defaulting to 8. Higher rank values increase capacity but consume more GPU memory.
c. lora.alpha: Accepts an integer value as the scaling coefficient, equivalent to setting the LoRA amplification ratio, defaulting to 16.
d. lora.target_modules: Accepts a list[str] value, containing regex patterns to match modules where LoRA should be injected, applied to linear layers, defaulting to '^fs2\.' and '^diffusion\.'.
e. lora.base_ckpt: Accepts the file path to the base model for LoRA.
f. lora.train_bias: Accepts a boolean value, defaulting to True, indicating that biases of all linear layers are trained alongside LoRA. Recommended for small datasets (<5 minutes).
g. lora.merge_before_export: Accepts a boolean value, defaulting to True, indicating whether to merge LoRA components into the base model before ONNX export.