Compiling code for WikiData_counterfact and Wikidata_recent

[msumitaml]

How can we compile the code for WikiData_counterfact and Wikidata_recent for Llama, gpt2-xl, gptj and other models like shown for zsre dataset. How do we do it for WikiData_counterfact and Wikidata_recent and from where do we download the data?

[littlefive5]

Hello, we release the data we used in huggingface, you can see it in our README.md and the example in https://github.com/zjunlp/EasyEdit/blob/main/examples/KnowEdit.md

[msumitaml]

Thank you for the reply.

When I am trying to run this code it is giving me Assertion error

python run_knowedit_llama2.py --editing_method=ROME --hparams_dir=./hparams/ROME/llama-7b.yaml --data_dir=./data/wiki_counterfact/test_cf.json --datatype='counterfact'

2024-01-20 12:05:57,321 - easyeditor.editors.editor - INFO - Instantiating model
01/20/2024 12:05:57 - INFO - easyeditor.editors.editor - Instantiating model
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00, 1.25s/it]
One Edit instance needs one locality input.....
Traceback (most recent call last):
File "/EasyEdit-main/examples/run_knowedit_llama2.py", line 235, in
metrics, edited_model, _ = editor.edit(
File "/EasyEdit-main/examples/../easyeditor/editors/editor.py", line 156, in edit
requests = self._prepare_requests(prompts, target_new, ground_truth, rephrase_prompts,
File "/EasyEdit-main/examples/../easyeditor/editors/editor.py", line 508, in _prepare_requests
assert len(locality_inputs[locality_key]['prompt']) == len(locality_inputs[locality_key]['ground_truth'])
AssertionError

I have data in this format
data-->wiki_counterfact --> test_cf.json & train_cf.json

Also for data_dir, instead of data_dir=./data/wiki_counterfact/test_cf.json if I give data_dir=./data/ it gives me "Is a directory error". Using test_cf.json, gives assertion error.

What do I do to resolve the error and get the result?

[XeeKee]

please pull the newest code, we have fix this bug.

[msumitaml]

I am now working with the latest code and when running

python run_knowedit_llama2.py --editing_method=ROME --hparams_dir=/EasyEdit-main/hparams/ROME/llama-7b.yaml --data_dir=./data/wiki_counterfact/test_cf.json --datatype='counterfact'

This is generating the below error

Traceback (most recent call last):
File "/EasyEdit-main/examples/run_knowedit_llama2.py", line 73, in
an=pr["ground_truth"][0][0]
IndexError: list index out of range

[XeeKee]

I checked and found that the list at this location is empty. Could you manually add an empty string to the data with 'subject': 'Hong Chau'? We may update the dataset after our discussion.

[tbozhong]

We've just updated the datasets on Hugging Face. Please download the latest version to ensure compatibility.
Thank you for bringing this to our attention!

[zxlzr]

Hi, have you solved your issue?

[msumitaml]

We've just updated the datasets on Hugging Face. Please download the latest version to ensure compatibility. Thank you for bringing this to our attention!

You are welcome! Thank you so much for the super quick fixes.

I have tried just with ROME for Llama2-7b. Yes the issue has been resolved. Thank you!