New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TemplateLM#_encode_pair() only works for HF transformers auto-models #1966

Closed

Birch-san opened this issue Jun 14, 2024 · 1 comment

Birch-san commented Jun 14, 2024 •

edited

Loading

I invoked lm_eval.evaluator.evaluate(harness) with a harness which subclasses TemplateLM, on 0.4.2.

The text was updated successfully, but these errors were encountered:

Author

Birch-san commented Jun 14, 2024

okay the else branch in latest commit looks like it'll fix it fine.

lm-evaluation-harness/lm_eval/api/model.py

Line 352 in 568af94

return context_enc, continuation_enc

I thought I was running latest commit already, but I was misled by StackOverflow.

Birch-san closed this as completed

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment