Facing Issue While Running LM Eval

[VimalWill]

Hi Authors of Liger,

I extend my gratitude for your contribution to the field,

I'm trying to run the lm-eval for the checkpoints within the repo, and I'm ending up with an error. I attached the command and error I'm ending up with.

command:

  --model_args pretrained=~/Linearization/checkpoints/liger_qwen25_gla_base,trust_remote_code=True,model_type=causal \
    --tasks piqa,arc_easy,arc_challenge,hellaswag,winogrande \
    --batch_size 64 \
    --device cuda \
    --seed 0

Current Triton version 3.1.0 is below the recommended 3.2.0 version. Errors may occur and these issues will not be fixed. Please consider upgrading Triton.
2026-02-03:04:06:28 INFO     [_cli.run:376] Selected Tasks: ['piqa', 'arc_easy', 'arc_challenge', 'hellaswag', 'winogrande']
2026-02-03:04:06:28 INFO     [evaluator:211] Setting random seed to 0 | Setting numpy seed to 0 | Setting torch manual seed to 0 | Setting fewshot manual seed to 0
2026-02-03:04:06:28 INFO     [evaluator:236] Initializing hf model, with arguments: {'pretrained': '/home/vimal/phd_research/Linearization/checkpoints/liger_qwen25_gla_base', 'trust_remote_code': True, 'model_type': 'causal'}
2026-02-03:04:06:28 INFO     [models.huggingface:161] Using device 'cuda'
2026-02-03:04:06:28 INFO     [models.huggingface:548] Model type cannot be determined. Using default model type 'causal'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/vimal/phd_research/Linearization/eval/harness.py", line 23, in <module>
    cli_evaluate()
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/__main__.py", line 10, in cli_evaluate
    parser.execute(args)
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/_cli/harness.py", line 60, in execute
    args.func(args)
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/_cli/run.py", line 379, in _execute
    results = simple_evaluate(
              ^^^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/utils.py", line 498, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/evaluator.py", line 239, in simple_evaluate
    lm = lm_eval.api.registry.get_model(model).create_from_arg_obj(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/api/model.py", line 180, in create_from_arg_obj
    return cls(**arg_dict, **additional_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/models/huggingface.py", line 204, in __init__
    self._create_tokenizer(
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/lm_eval/models/huggingface.py", line 793, in _create_tokenizer
    self.tokenizer = transformers.AutoTokenizer.from_pretrained(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 937, in from_pretrained
    tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
                                               ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/vimal/phd_research/LinearEnv/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 760, in __getitem__
    model_type = self._reverse_config_mapping[key.__name__]
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'LigerQwen2GLAConfig'

Kindly please help me in this issue

[VimalWill]

I'm also facing this issues when I go further,

File "/home/u32/vimalwilliam/.conda/envs/liger/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 640, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/home/u32/vimalwilliam/.conda/envs/liger/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/u32/vimalwilliam/.conda/envs/liger/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/u32/vimalwilliam/Linearization/liger/models/liger_gla/modeling_liger_gla.py", line 123, in forward
    o_, recurrent_state = fused_chunk_gla(q, k, v, g, scale=scale, initial_state=recurrent_state, output_final_state=True)
TypeError: fused_chunk_gla() takes 0 positional arguments but 4 were given
Running loglikelihood requests:   0%|