Add version config option.

[PhilipMay]

see #172

[PhilipMay]

@clefourrier

You write:

and edit the task_table.jsonl to set "version" to 0

I do not understand why. When version has the default of 0 I do not need to change that file.
Only if someone changes the prompt or the underlying data the version can be increased.

What do you think?

[PhilipMay]

@clefourrier can you please review the PR. I added some TODO comments. They need to be checked.

@jphme maybe you also could have a quick look?

[clefourrier]

Hi @PhilipMay !
I think it's better to make explicit the task version in the json file, otherwise it will be too easy to forget to edit it afterwards.

[PhilipMay]

Hi @PhilipMay ! I think it's better to make explicit the task version in the json file, otherwise it will be too easy to forget to edit it afterwards.

Ahh I see. I made the change.

[PhilipMay]

Hi @PhilipMay ! I think it's better to make explicit the task version in the json file, otherwise it will be too easy to forget to edit it afterwards.

Should I also set the version to 0 in https://github.com/huggingface/lighteval/blob/main/community_tasks/arabic_evals.py ?

[clefourrier]

Yep, let's be exhaustive :)

[PhilipMay]

Yep, let's be exhaustive :)

I am done with that.

[clefourrier]

Nice! Just need to merge main in your branch and I think we'll be good to go

[clefourrier]

LGTM, thanks, let's merge once the tests pass