0.1.1

What's in this release:

• A simple wrapper for lm-eval (lm-evaluation-harness) so that it can be used easily and integrated in different LLM based workflows.
• Find the PyPI release here: https://pypi.org/project/easy-lm-eval/

What's Changed

• Initial PR for adding light-lm-eval package by @Anindyadeep in #1
• Workflow update by @Anindyadeep in #6

New Contributors

• @Anindyadeep made their first contribution in #1

Full Changelog: 0.0.2...0.0.0

Quick Usage:

from easy_eval import HarnessEvaluator, HarnessTaskManager
from easy_eval.config import EvaluatorConfig

tasks = HarnessTaskManager.load_tasks(["babi"])
print(tasks)
config = EvaluatorConfig(limit=10)

eval = HarnessEvaluator(model_backend="huggingface", model_name_or_path="gpt2")
results = eval.evaluate(tasks=tasks, show_results_terminal=True, config=config)

print(results)