Dataset download problems #165
Description
Hello!
I chose creating and evaluating a NAVSIM agent as my thesis work, which I believe is ready.
However, I have had problems with downloading and placing the dataset. The markdowns don't describe how the new scripts in download/ should be used, so I would like to ask you for help.
I ran download_navtrain_aws.sh. I placed the two folders that I got (navsim_logs and sensor_blobs) under dataset/navsim_logs and dataset/sensor_blobs with each renamed to trainval, as described in the markdowns. After that I ran download_test.sh and placed the result folder similarily, under dataset/navsim_logs and dataset/sensor_blobs with each renamed to test.
After this, I wanted to create the caches, so I ran run_metric_caching.sh two times: one time with TRAIN_TEST_SPLIT=navtest into exp/metric_cache and one time with TRAIN_TEST_SPLIT=navtrain into exp/training_cache.
I got the error message:
bash run_camera_only_agent_training.sh
Seed set to 0
[2025-10-02 00:18:29,996][main][INFO] - Global Seed set to 0
[2025-10-02 00:18:29,997][main][INFO] - Path where all results are stored: /home/hunor/thesis/exp/basemodel/2025.10.02.00.18.26
[2025-10-02 00:18:29,997][main][INFO] - Building Agent
Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
Some weights of ViTModel were not initialized from the model checkpoint at facebook/dino-vits8 and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
/conda/anaconda3/envs/navsim/lib/python3.9/site-packages/torch/nn/modules/transformer.py:286: UserWarning: enable_nested_tensor is True, but self.use_nested_tensor is False because encoder_layer.self_attn.batch_first was not True(use batch_first for better inference performance)
warnings.warn(f"enable_nested_tensor is True, but self.use_nested_tensor is False because {why_not_sparsity_fast_path}")
[2025-10-02 00:18:30,557][main][INFO] - Building Lightning Module
[2025-10-02 00:18:30,557][main][INFO] - Using cached data without building SceneLoader
Loading Valid Caches: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 978/978 [00:00<00:00, 81899.72it/s]
Loading Valid Caches: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 214/214 [00:00<00:00, 83635.95it/s]
[2025-10-02 00:18:30,643][main][INFO] - Building Datasets
Error executing job with overrides: ['agent=camera_only_agent', 'experiment_name=basemodel', 'trainer.params.max_epochs=50', 'train_test_split=navtrain']
Traceback (most recent call last):
File "/home/hunor/thesis/my-navsim/navsim/planning/script/run_training.py", line 133, in main
train_dataloader = DataLoader(train_data, **cfg.dataloader.params, shuffle=True)
File "/conda/anaconda3/envs/navsim/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 350, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File "/conda/anaconda3/envs/navsim/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 143, in init
raise ValueError(f"num_samples should be a positive integer value, but got num_samples={self.num_samples}")
ValueError: num_samples should be a positive integer value, but got num_samples=0
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
I don't whether I made a mistake in placing the datasets or with using the scripts or wrong scripts.
Please help me overcome the issue!