CI fails with dev dependencies: TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'

[albertvillanova]

CI fails with dev dependencies: https://github.com/huggingface/trl/actions/runs/18493152127/job/52691262212

TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'

FAILED tests/test_bco_trainer.py::TestBCOTrainer::test_lora_train_and_save - TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'
FAILED tests/test_kto_trainer.py::TestKTOTrainer::test_kto_lora_save - TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'
FAILED tests/test_dpo_trainer.py::TestDPOTrainer::test_dpo_lora_save - TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'

Stacktrace:

 >       AutoModelForCausalLM.from_pretrained(self.tmp_dir)

tests/test_bco_trainer.py:409: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12/site-packages/transformers/models/auto/auto_factory.py:385: in from_pretrained
    return model_class.from_pretrained(
.venv/lib/python3.12/site-packages/transformers/modeling_utils.py:273: in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.12/site-packages/transformers/modeling_utils.py:4662: in from_pretrained
    model.load_adapter(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Qwen2ForCausalLM(
  (model): Qwen2Model(
    (embed_tokens): Embedding(151665, 8)
    (layers): ModuleList(
      (0-1...-06)
    (rotary_emb): Qwen2RotaryEmbedding()
  )
  (lm_head): Linear(in_features=8, out_features=151665, bias=False)
)
peft_model_id = '/tmp/pytest-of-root/pytest-0/popen-gw2/test_lora_train_and_save5'
adapter_name = 'default', revision = None, token = None, device_map = 'auto'
max_memory = None, offload_folder = None, offload_index = None
peft_config = None, adapter_state_dict = None, low_cpu_mem_usage = False
is_trainable = False
adapter_kwargs = {'_adapter_model_path': '/tmp/pytest-of-root/pytest-0/popen-gw2/test_lora_train_and_save5'}

    def load_adapter(
        self,
        peft_model_id: Optional[str] = None,
        adapter_name: Optional[str] = None,
        revision: Optional[str] = None,
        token: Optional[str] = None,
        device_map: str = "auto",
        max_memory: Optional[str] = None,
        offload_folder: Optional[str] = None,
        offload_index: Optional[int] = None,
        peft_config: Optional[dict[str, Any]] = None,
        adapter_state_dict: Optional[dict[str, "torch.Tensor"]] = None,
        low_cpu_mem_usage: bool = False,
        is_trainable: bool = False,
        adapter_kwargs: Optional[dict[str, Any]] = None,
    ) -> None:
        """
        Load adapter weights from file or remote Hub folder. If you are not familiar with adapters and PEFT methods, we
        invite you to read more about them on PEFT official documentation: https://huggingface.co/docs/peft
    
        Requires PEFT to be installed as a backend to load the adapter weights.
    
        Args:
            peft_model_id (`str`, *optional*):
                The identifier of the model to look for on the Hub, or a local path to the saved adapter config file
                and adapter weights.
            adapter_name (`str`, *optional*):
                The adapter name to use. If not set, will use the name "default".
            revision (`str`, *optional*, defaults to `"main"`):
                The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
                git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
                identifier allowed by git.
    
                > [!TIP]
                > To test a pull request you made on the Hub, you can pass `revision="refs/pr/<pr_number>"`.
    
            token (`str`, `optional`):
                Whether to use authentication token to load the remote folder. Useful to load private repositories
                that are on HuggingFace Hub. You might need to call `hf auth login` and paste your tokens to
                cache it.
            device_map (`str` or `dict[str, Union[int, str, torch.device]]` or `int` or `torch.device`, *optional*):
                A map that specifies where each submodule should go. It doesn't need to be refined to each
                parameter/buffer name, once a given module name is inside, every submodule of it will be sent to the
                same device. If we only pass the device (*e.g.*, `"cpu"`, `"cuda:1"`, `"mps"`, or a GPU ordinal rank
                like `1`) on which the model will be allocated, the device map will map the entire model to this
                device. Passing `device_map = 0` means put the whole model on GPU 0.
    
                To have Accelerate compute the most optimized `device_map` automatically, set `device_map="auto"`. For
                more information about each option see [designing a device
                map](https://hf.co/docs/accelerate/main/en/usage_guides/big_modeling#designing-a-device-map).
            max_memory (`Dict`, *optional*):
                A dictionary device identifier to maximum memory. Will default to the maximum memory available for each
                GPU and the available CPU RAM if unset.
            offload_folder (`str` or `os.PathLike`, `optional`):
                If the `device_map` contains any value `"disk"`, the folder where we will offload weights.
            offload_index (`int`, `optional`):
                `offload_index` argument to be passed to `accelerate.dispatch_model` method.
            peft_config (`dict[str, Any]`, *optional*):
                The configuration of the adapter to add, supported adapters are all non-prompt learning configs (LoRA,
                IA³, etc). This argument is used in case users directly pass PEFT state dicts.
            adapter_state_dict (`dict[str, torch.Tensor]`, *optional*):
                The state dict of the adapter to load. This argument is used in case users directly pass PEFT state
                dicts.
            low_cpu_mem_usage (`bool`, *optional*, defaults to `False`):
                Reduce memory usage while loading the PEFT adapter. This should also speed up the loading process.
                Requires PEFT version 0.13.0 or higher.
            is_trainable (`bool`, *optional*, defaults to `False`):
                Whether the adapter should be trainable or not. If `False`, the adapter will be frozen and can only be
                used for inference.
            adapter_kwargs (`dict[str, Any]`, *optional*):
                Additional keyword arguments passed along to the `from_pretrained` method of the adapter config and
                `find_adapter_config_file` method.
        """
        check_peft_version(min_version=MIN_PEFT_VERSION)
    
        # peft only supports low_cpu_mem_usage starting from v0.13.0
        peft_load_kwargs = {}
        key_mapping = adapter_kwargs.pop("key_mapping", None) if adapter_kwargs is not None else None
        if key_mapping is None and any(allowed_name in self.__class__.__name__.lower() for allowed_name in VLMS):
            key_mapping = self._checkpoint_conversion_mapping
        if low_cpu_mem_usage:
            min_version_lcmu = "0.13.0"
            if version.parse(importlib.metadata.version("peft")) >= version.parse(min_version_lcmu):
                peft_load_kwargs["low_cpu_mem_usage"] = low_cpu_mem_usage
            else:
                raise ValueError(
                    "The version of PEFT you are using does not support `low_cpu_mem_usage` yet, "
                    f"please install PEFT >= {min_version_lcmu}."
                )
    
        adapter_name = adapter_name if adapter_name is not None else "default"
        if adapter_kwargs is None:
            adapter_kwargs = {}
    
        from peft import PeftConfig, inject_adapter_in_model, load_peft_weights
        from peft.utils import set_peft_model_state_dict
    
        if self._hf_peft_config_loaded and adapter_name in self.peft_config:
            raise ValueError(f"Adapter with name {adapter_name} already exists. Please use a different name.")
    
        if peft_model_id is None and (adapter_state_dict is None and peft_config is None):
            raise ValueError(
                "You should either pass a `peft_model_id` or a `peft_config` and `adapter_state_dict` to load an adapter."
            )
    
        if "device" not in adapter_kwargs:
            device = self.device if not hasattr(self, "hf_device_map") else list(self.hf_device_map.values())[0]
        else:
            device = adapter_kwargs.pop("device")
    
        # To avoid PEFT errors later on with safetensors.
        if isinstance(device, torch.device):
            device = str(device)
    
        # We keep `revision` in the signature for backward compatibility
        if revision is not None and "revision" not in adapter_kwargs:
            adapter_kwargs["revision"] = revision
        elif revision is not None and "revision" in adapter_kwargs and revision != adapter_kwargs["revision"]:
            logger.error(
                "You passed a `revision` argument both in `adapter_kwargs` and as a standalone argument. "
                "The one in `adapter_kwargs` will be used."
            )
    
        # Override token with adapter_kwargs' token
        if "token" in adapter_kwargs:
            token = adapter_kwargs.pop("token")
    
        if peft_config is None:
>           adapter_config_file = find_adapter_config_file(
                peft_model_id,
                token=token,
                **adapter_kwargs,
            )
E           TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path'

.venv/lib/python3.12/site-packages/transformers/integrations/peft.py:221: TypeError

[albertvillanova]

After investigation, it seems this PR in transformers is the root cause:

• [from_pretrained] Small refactor from_pretrained: move around unrelated stuff transformers#41445

[albertvillanova]

After further investigation, I think a bug was introduced in transformers with the above mentioned PR:

• [from_pretrained] Small refactor from_pretrained: move around unrelated stuff transformers#41445

The adapter_kwargs is passed to maybe_load_adapters as unpacked keyword arguments: Python does not pass the original dictionary, but instead creates a new dictionary inside the function; so the pop inside the function does not have the intended behavior.

def maybe_load_adapters(
    pretrained_model_name_or_path,
    download_kwargs: DownloadKwargs,
    **adapter_kwargs,
):
    ...
    _adapter_model_path = adapter_kwargs.pop("_adapter_model_path", None)

[albertvillanova]

I opened a PR in transformers to address this issue:

• Fix TypeError: find_adapter_config_file() got an unexpected keyword argument '_adapter_model_path' transformers#41604