Action Detection Testing with Batch Size > 1 -> AssertionError: only accept 1 sample at test mode

[bhimar]

When I increase the batch size for testing/validation in a config for spatio-temporal action detection, I run into the following error:
AssertionError: only accept 1 sample at test mode

Question: How do I run testing/validation with batch size > 1 for spatio-temporal action detection tasks?

Details
I am using a config based on the following config that is given in the mmaction2 codebase: configs/detection/ava/slowfast_kinetics_pretrained_r50_4x16x1_20e_ava_rgb.py

In this config I modify the videos_per_gpu field for the val and test dataloaders in the data dictionary (starting on line 120) in the config:

data = dict(  
    videos_per_gpu=9,  
    workers_per_gpu=2,  
    val_dataloader=dict(videos_per_gpu=2), # increase batch size from 1 to 2  
    test_dataloader=dict(videos_per_gpu=2), # increased batch size from 1 to 2  
    train=dict(  
        type=dataset_type,  
        ann_file=ann_file_train,  
        exclude_file=exclude_file_train,  
        pipeline=train_pipeline,  
        label_file=label_file,  
        proposal_file=proposal_file_train,  
        person_det_score_thr=0.9,  
        data_prefix=data_root),  
    val=dict(  
        type=dataset_type,  
        ann_file=ann_file_val,  
        exclude_file=exclude_file_val,  
        pipeline=val_pipeline,  
        label_file=label_file,  
        proposal_file=proposal_file_val,  
        person_det_score_thr=0.9,  
        data_prefix=data_root))  
data['test'] = data['val']  

When I run the testing code (tools/test.py) with this config, I run into the AssertionError.
Help is much appreciated!