DisAlign

DisAlign is a toolkit for training multiple expert adapters from a shared base model and merging them with a geometry-aware DisAlign workflow.

This repository currently contains:

• DisAlign merge implementation
• expert dataset preparation helpers for HH-RLHF-style data
• LoRA expert training scripts
• example YAML configs for single-expert and multi-expert merging
• documentation for training, saving, and merging experts

Main Entry Points

Use the new disalign package and CLI names:

• python -m disalign.scripts.disalign
• python -m disalign.scripts.disalign_prepare_hhh_data
• python -m disalign.scripts.disalign_train_lora_expert

If installed as console scripts:

• disalign
• disalign-prepare-hhh
• disalign-train-lora-expert

Older mergekit.* imports and mergekit-* CLI names are kept only as compatibility aliases.

Documentation

• Main guide: docs/disalign_README.md
• HHH pipeline guide: docs/disalign_hhh_readme.md
• Merge-method notes: docs/merge_methods.md

Typical Workflow

1. Prepare one dataset per expert objective.
2. Train one LoRA expert per dataset.
3. Write a DisAlign YAML that lists all experts.
4. Run the DisAlign merge script.

Example Files

• Multi-expert HHH config: examples/disalign_hhh_llama32_1b_lora.yml
• Local single-expert config: examples/disalign_local_llama32_1b.yml
• HHH end-to-end script: examples/disalign_hhh_llama32_1b.sh
• Single-expert smoketest config: examples/disalign_hhh_smoketest_single_expert.yml

Installation

git clone https://github.com/erzhoujk/DisAlign.git disalign
cd disalign
pip install -e .

Notes

• The current codebase still includes parts inherited from the original mergekit structure, but the primary project-facing naming has been switched to disalign.
• The recommended package path is disalign, not mergekit.