You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for your great work.
Following the steps, I am up to running the following command for CNNDM datasets, python data_prepro_clean.py --mode bpe_binarize --input_dir <my_processed-data-dir> --tokenizer_dir <my_bpe-dir>
but got the following error,
Traceback (most recent call last):
File "../fairseq_cli/preprocess.py", line 452, in
cli_main()
File "../fairseq_cli/preprocess.py", line 448, in cli_main
main(args)
File "../fairseq_cli/preprocess.py", line 331, in main
make_all(args.source_lang, src_dict)
File "../fairseq_cli/preprocess.py", line 301, in make_all
make_dataset(vocab, args.trainpref, "train", lang, num_workers=args.workers)
File "../fairseq_cli/preprocess.py", line 297, in make_dataset
make_binary_dataset(vocab, input_prefix, output_prefix, lang, num_workers)
File "../fairseq_cli/preprocess.py", line 173, in make_binary_dataset
100 * sum(replaced.values()) / n_seq_tok[1],
ZeroDivisionError: division by zero
Could you please help? Thanks
The text was updated successfully, but these errors were encountered:
Thanks for your great work.
Following the steps, I am up to running the following command for CNNDM datasets,
python data_prepro_clean.py --mode bpe_binarize --input_dir <my_processed-data-dir> --tokenizer_dir <my_bpe-dir>
but got the following error,
Could you please help? Thanks
The text was updated successfully, but these errors were encountered: