Describe the bug
Metric:call used "&".join(f"{k}={v}" for k, v in sample_params.items()) to update metric name, but the value might be function, which will change for each run.
To Reproduce
Task that use sample_params, such as:
leaderboard:gsm8k:0
metrics=[
Metrics.exact_match(sample_params={"normalize_gold": gsm8k_normalizer, "normalize_pred": gsm8k_normalizer})
],
Expected behavior
Cache should work
Version info
main branch