ReferenceError: weakly-referenced object no longer exists

[kargarisaac]

Hello,

I'm trying to run the code form the closed_loop_test.ipynb notebook locally on my m1 macbook pro. But it gives me the following error. I tested some versions of protobuf but it didn't help. What do you think is the problem?

I downloaded the dataset and edited the config file.

(l5kit) ➜  urban_driver python closed_loop_test.py
/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/site-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)
  warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"
+------------+------------+------------+---------------+-----------------+----------------------+----------------------+----------------------+---------------------+
| Num Scenes | Num Frames | Num Agents | Num TR lights | Total Time (hr) | Avg Frames per Scene | Avg Agents per Frame | Avg Scene Time (sec) | Avg Frame frequency |
+------------+------------+------------+---------------+-----------------+----------------------+----------------------+----------------------+---------------------+
|   16220    |  4030296   | 312617887  |    29277930   |      111.97     |        248.48        |        77.57         |        24.85         |        10.00        |
+------------+------------+------------+---------------+-----------------+----------------------+----------------------+----------------------+---------------------+
/Users/isaackargar/codes/phd/l5kit/l5kit/l5kit/data/zarr_dataset.py:213: RuntimeWarning: zarr dataset path should end with .zarr (for now). Open will fail for this dataset!
  dataset = ChunkedDataset("")
/Users/isaackargar/codes/phd/l5kit/l5kit/l5kit/simulation/utils.py:107: RuntimeWarning: zarr dataset path should end with .zarr (for now). Open will fail for this dataset!
  new_dataset = ChunkedDataset("")
Traceback (most recent call last):
  File "/Users/isaackargar/codes/phd/urban_driver/closed_loop_test.py", line 108, in <module>
    sim_outs = sim_loop.unroll(scenes_to_unroll)
  File "/Users/isaackargar/codes/phd/l5kit/l5kit/l5kit/simulation/unroll.py", line 180, in unroll
    sim_dataset = SimulationDataset.from_dataset_indices(self.dataset, scene_indices, self.sim_cfg)
  File "/Users/isaackargar/codes/phd/l5kit/l5kit/l5kit/simulation/dataset.py", line 111, in from_dataset_indices
    return SimulationDataset(scene_dataset_batch, sim_cfg)
  File "/Users/isaackargar/codes/phd/l5kit/l5kit/l5kit/simulation/dataset.py", line 88, in __init__
    self.recorded_scene_dataset_batch = deepcopy(self.scene_dataset_batch)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 211, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 211, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/isaackargar/miniconda3/envs/l5kit/lib/python3.10/copy.py", line 151, in deepcopy
    copier = getattr(x, "__deepcopy__", None)
ReferenceError: weakly-referenced object no longer exists

[kargarisaac]

Solved:

changed the code in l5kit\l5kit\l5kit\simulation\dataset.py\SimulationDataset\_init__ to use a custom function from here:

def deepcopy2(obj):
            if isinstance(obj, dict):
                return {deepcopy2(key): deepcopy2(value) for key, value in obj.items()}
            if hasattr(obj, '__iter__'):
                return type(obj)(deepcopy2(item) for item in obj)
            return obj
            
        self.recorded_scene_dataset_batch = deepcopy2(self.scene_dataset_batch)