Errors in calculating EPDMS

[zhexilian]

Hi authors @VVeiCao @kashyap7x

Thanks for your contribution to the community!

I have downloaded datasets and built the NAVSIM 2.2 environment step-by-step following the instruction Docs. And I want to evaluate the human agent's EPDMS at first. I have ran the following command:

TRAIN_TEST_SPLIT=navtest
CACHE_PATH=$NAVSIM_EXP_ROOT/metric_cache

python $NAVSIM_DEVKIT_ROOT/navsim/planning/script/run_metric_caching.py \
train_test_split=$TRAIN_TEST_SPLIT \
metric_cache_path=$CACHE_PATH

TRAIN_TEST_SPLIT=navtest
CACHE_PATH=$NAVSIM_EXP_ROOT/metric_cache

python $NAVSIM_DEVKIT_ROOT/navsim/planning/script/run_pdm_score_one_stage.py \
train_test_split=$TRAIN_TEST_SPLIT \
agent=human_agent \
experiment_name=human_agent \
metric_cache_path=$CACHE_PATH \

However, ERRORS seem to exsit in the evaluation output:

2025.08.06.15.54.21.csv

I didn't change anything of the codebase. Have you ever met this issure before?

[zhexilian]

After a lot of tries, I finally found where the bug is:

• I thought the human agent evaluation didn't need any GPUs so I only evaluated it through CPUs.
• The evaluation process didn't raise any traceback errors but the evaluation result seems to be wrong.
• Finally I run the evaluation with an L40 GPU and the evaluation result returns to correct.

If the authors have seen the comment, you can close this issue. Thank you very much.