verbalizers for trec dataset

[dorost1234]

Hi
this seems to me the place of "location" and "number" needs to get swapped, as label 4 (zero-indexed) corresponds to "number" and label "5" corresponds to "location".

Here is the link to the dataset labels: https://huggingface.co/datasets/viewer/ if you search trec

Here are the current verbalizers: ["Description", "Entity", "Expression", ​"Human", "Location", "Number"]

thanks

[shmsw25]

Hi @dorost1234,

We did not use the Huggingface version of the data. Instead, we downloaded the data from here, and I believe two versions of the data have different label ordering.

I manually checked the examples and double-checked that the current verbalizers are correct. For example, label "4" includes

• What continent's name appears on the upper left corner of a Budweiser label?
• What European city do Nicois live in?
• Where is the Isle of Man?

(all location related)

label "5" includes

• How many bails are there in a cricket wicket?
• At what age did Rossini stop writing opera?
• At was the first minimum wage?

(all number related)