fix multi lora training

[tastelikefeet]

PR type

• Bug Fix
• New Feature
• Document Updates
• More Models or Datasets Support

PR information

Write the detail information belongs to this PR.

Experiment results

Paste your experiment result here(if needed).

[gemini-code-assist]

Code Review

This pull request introduces an optimizer_context to isolate parameters for specific LoRA adapters during optimization in Megatron and ensures all LoRA parameters are marked as trainable to support MegatronDDP gradient buffers. It also includes a safety check for missing labels in the input feature template. Review feedback suggests making the use_distributed_optimizer setting configurable rather than hardcoded to avoid silently disabling performance features and refining the regex pattern in the optimizer context to handle parameter names that may not start with a leading dot.