Search results

Max token issue on RULER

prio

science-team

Status: Open.

#819 In huggingface/lighteval;

· clefourrier opened on Jun 20, 2025

Mamba OOM deep dive

prio

science-team

Status: Open.

#818 In huggingface/lighteval;

· clefourrier opened on Jun 20, 2025

[EVAL] Needle in a haystack

new-task

science-team

Status: Open.

#817 In huggingface/lighteval;

· clefourrier opened on Jun 20, 2025

[EVAL] LVEval

new-task

science-team

Status: Open.

#816 In huggingface/lighteval;

· clefourrier opened on Jun 20, 2025

[EVAL] MRCR

new-task

science-team

Status: Open.

#815 In huggingface/lighteval;

· clefourrier opened on Jun 20, 2025

[BUG] Failed downloads / 429 rate limiting for concurrent evaluations sharing cache

bugSomething isn't workingSomething isn't working

Status: Open.

#813 In huggingface/lighteval;

· lukemerrick opened on Jun 16, 2025

[FT] VL model support for vllm/sglang backend?

Status: Open.

#808 In huggingface/lighteval;

· xgwang opened on Jun 11, 2025

[FT] Allow configuration of <code>litellm</code> cache location

Status: Open.

#806 In huggingface/lighteval;

· jamesbraza opened on Jun 10, 2025

[BUG] <code>Registry.task_registry</code> not reimporting the custom task module

bugSomething isn't workingSomething isn't working

Status: Open.

#805 In huggingface/lighteval;

· jamesbraza opened on Jun 10, 2025

[FT] showing count in Markdown summary table

feature/enhancementNew feature/requestNew feature/request

good first issueGood for newcomersGood for newcomers

Status: Open.

Feature

#804 In huggingface/lighteval;

· jamesbraza opened on Jun 9, 2025

[FT] <code>MetricUseCase.FORMATTING</code> for reasoning models

Status: Open.

#803 In huggingface/lighteval;

· jamesbraza opened on Jun 9, 2025

[BUG] Unable to Download Dataset in Restricted Network Environment (No Access to Hugging Face or Mirrors)

bugSomething isn't workingSomething isn't working

Status: Open.

#802 In huggingface/lighteval;

· xjli360 opened on Jun 9, 2025