Error in Reading file #3
Description
Hey, I have downloaded dataset input v1 and unzip them into input dir and run python run.py with all recommanded parameters. However, there is a strange error show that the dataset file cannot be read correctly. Here is the output:
Traceback (most recent call last):
File "$HOME/.pycharm_helpers/pydev/pydevd.py", line 1664, in <module>
main()
File "$HOME/.pycharm_helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "$HOME/.pycharm_helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "$HOME/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "$HOME/Code/PointerSQL/run.py", line 117, in <module>
NL2Prog_meta.run_wikisql(input_dir, output_dir, config, model_to_test)
File "$HOME/Code/PointerSQL/app/nl2prog_meta.py", line 268, in run_wikisql
test_support_file=test_support_file)
File "$HOME/Code/PointerSQL/app/nl2prog_meta.py", line 58, in __init__
train_dataset, train_support_dataset = app.util.read_dataset(train_file, support_file_list=train_support_file, input_num=2)
File "$HOME/Code/PointerSQL/app/util.py", line 34, in read_dataset
output_num=output_num) for _filename in support_file_list]
File "$HOME/Code/PointerSQL/app/util.py", line 34, in <listcomp>
output_num=output_num) for _filename in support_file_list]
File "$HOME/Code/PointerSQL/app/util.py", line 7, in _read_dataset
raw_lines = f.readlines()
File "$HOME/anaconda3/envs/py36-tf14/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 4046: ordinal not in range(128)
BTW, I figured out the filename causing this issue is wikisql_train_support_rank_0