Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Erro!!! #8

Open
little-black-f opened this issue Nov 28, 2020 · 1 comment
Open

Erro!!! #8

little-black-f opened this issue Nov 28, 2020 · 1 comment

Comments

@little-black-f
Copy link

Why can't I find the canonical_data file in MNLI, and when running the example provided in the webpage, I got the following error:

AssertionError Traceback (most recent call last)
/data-tmp/TM-DNN/MT-DNN-master/mtdnn/data_builder_mtdnn.py in load_and_build_data(self, dump_rows)
131 task_load_func = self.supported_tasks_loader_map[name]
--> 132 data = task_load_func(in_file_path, data_opts)
133 processed_rows = process_data_and_dump_rows(

/data-tmp/TM-DNN/MT-DNN-master/mtdnn/tasks/utils.py in load_mnli(file_path, kwargs)
125 blocks = line.strip().split("\t")
--> 126 assert len(blocks) > 9
127 if blocks[-1] == "-":

AssertionError:

During handling of the above exception, another exception occurred:

OSError Traceback (most recent call last)
in
5 data_dir=DATA_SOURCE_DIR_MNLI,
6 canonical_data_suffix="canonical_data",
----> 7 dump_rows=True,
8 )
9

/data-tmp/TM-DNN/MT-DNN-master/mtdnn/data_builder_mtdnn.py in init(self, tokenizer, task_defs, do_lower_case, data_dir, canonical_data_suffix, dump_rows)
196 )
197 self.processed_tasks_data = self.task_data_loader.load_and_build_data(
--> 198 self.save_to_file
199 )
200

/data-tmp/TM-DNN/MT-DNN-master/mtdnn/data_builder_mtdnn.py in load_and_build_data(self, dump_rows)
145 )
146 except Exception as ex:
--> 147 raise IOError(ex)
148 return processed_data
149

OSError:

which happened in :
Data Preprocessing
Create the Data Builder Object

Is this related to my pytorch using version 1.5.0?

@matteoghera
Copy link

Your MNLI data is corrupt. You can try to change assert with this if:
if len(blocks) > 9:
continue
This omits the corrupt line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants