Error running uniform partition for text classification

[bangawayoo]

Hi.
I am encountering a EOFError when trying to run uniform partition for text classification.

run_text_classification.sh FedOPT "uniform" 5e-5 0.1 51 4

27440 2022-01-09,13:28:37.575 - {base_data_manager.py (306)} - _load_data_loader_from_cache(): Loading features from cached file cache_dir/distilbert_distilbert-base-uncased_cached_256_ClassificationModel_20news_uniform_75
Traceback (most recent call last):
File "/home/ky/miniconda3/envs/fednlp/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/ky/miniconda3/envs/fednlp/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/ky/Research/NLP/FL/FedNLP/experiments/distributed/transformer_exps/run_tc_exps/fedavg_main_tc.py", line 140, in
train_data_local_dict, test_data_local_dict, num_clients = dm.load_federated_data(process_id=process_id)
File "/home/ky/Research/NLP/FL/FedNLP/data_manager/base_data_manager.py", line 142, in load_federated_data
return self._load_federated_data_local()
File "/home/ky/Research/NLP/FL/FedNLP/data_manager/base_data_manager.py", line 240, in _load_federated_data_local
state, res = self._load_data_loader_from_cache(client_idx)
File "/home/ky/Research/NLP/FL/FedNLP/data_manager/base_data_manager.py", line 309, in _load_data_loader_from_cache
train_examples, train_features, train_dataset, test_examples, test_features, test_dataset = pickle.load(handle)
EOFError: Ran out of input

Non-iid partition method works fine.
Any suggestion on how to fix this?

Thanks.

[bangawayoo]

Seems relevant to #16.
Following the below solution fixes this.

please add --reprocess_input_data at the end of the bash script