Merging non gptq adapter to gptq model

[flozi00]

System Info

master branch

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

start launcher with gptq model, then try to load non gptq adapter

2023-11-22T19:34:06.185127Z ERROR lorax_client: router/client/src/lib.rs:33: Server error: 'QuantLinear' object has no attribute 'weight'

for tests if commented out the ID check

#if adapter_config.base_model_name_or_path != model_id:
    #    raise ValueError(f"Adapter '{adapter_id}' is not compatible with model '{model_id}'. "
    #                        f"Use --model-id '{adapter_config.base_model_name_or_path}' instead.")

I am already thinking about better check, because it depends on model arch and parameters count instead of the id directly. so zephyr lora could be merged successfully to mistral instruct too.

Expected behavior

detect gptq model and use qweight instead of weight when merging

[flozi00]

@tgaddair open for discussion ?

[tgaddair]

Absolutely! @flozi00, can you point me to the base model and adapter you're using to help with the repro?

[flozi00]

Base model for example
https://huggingface.co/flozi00/Mistral-7B-german-assistant-v5-4bit-autogptq

Mistral 7b arch

The adapter is from the examples in readme
Just copied to readme codeblock for testing

[tgaddair]

Ah yeah, definitely agree the current check is too aggressive here. I was thinking of changing it to be a warning (like: adapter was trained on a different base model with the same architecture). I can take a quick look.