AttributeError: 'MyDataset' object has no attribute 'global_rank'

[cahya-wirawan]

Hi,

When I run the train.py script of RWKV-v5, I get error message AttributeError: 'MyDataset' object has no attribute 'global_rank' as follow:

AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/cahya/miniconda3/envs/rwkv/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/cahya/miniconda3/envs/rwkv/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/cahya/miniconda3/envs/rwkv/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/cahya/Work/RWKV-LM/RWKV-v5/src/dataset.py", line 104, in __getitem__
    rank = self.global_rank
AttributeError: 'MyDataset' object has no attribute 'global_rank'

my torch version is: 2.1.0.dev20230629

[cahya-wirawan]

The problem was caused by the pytorch lightning version 2.1.1. After I downgraded to 1.9.5, the problem was solved.

[DarokCx]

followed those instructions but now: argument of type 'NoneType' is not iterable happens

[BlinkDL]

@DarokCx IMPORTANT: Use deepspeed==0.7.0 pytorch-lightning==1.9.5 torch 1.13.1+cu117