Mismatching path in Windows when loading UCF101

[xianyuanliu]

🐛 Describe the bug

I tried to load UCF-101 by torchvision.datasets.UCF101 on my Windows laptop. However, I cannot get correct indices by _select_fold. Line 113 may go wrong in Windows.

Variable i is 'path_to_video/ApplyEyeMakeup\\v_ApplyEyeMakeup_g01_c01.avi', while
selected_files is 'path_to_video/ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01.avi'

The reason is that i is created by os.path.joint() using the double right slash. But selected_files is extracted from annotation files. Paths in these files are using the left slash, e.g. 'ApplyEyeMakeup/v_ApplyEyeMakeup_g01_c01.avi'.

May I know how to fix this, please? Many thanks!

Versions

Collecting environment information...
PyTorch version: 1.10.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.18363-SP0
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\cudnn64_7.dll
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy==0.910
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.4
[pip3] pytorch-lightning==1.5.5
[pip3] pytorch-memlab==0.2.4
[pip3] torch==1.10.0+cu102
[pip3] torch-geometric==2.0.2
[pip3] torch-scatter==2.0.9
[pip3] torch-sparse==0.6.12
[pip3] torchaudio==0.10.0+cu102
[pip3] torchmetrics==0.6.1
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.11.1+cu102
[conda] libblas 3.9.0 12_win64_mkl conda-forge
[conda] libcblas 3.9.0 12_win64_mkl conda-forge
[conda] liblapack 3.9.0 12_win64_mkl conda-forge
[conda] mkl 2021.4.0 h0e2418a_729 conda-forge
[conda] mypy 0.910 pypi_0 pypi
[conda] mypy-extensions 0.4.3 pypi_0 pypi
[conda] numpy 1.21.4 py38h089cfbf_0 conda-forge
[conda] pytorch-lightning 1.5.5 pypi_0 pypi
[conda] pytorch-memlab 0.2.4 pypi_0 pypi
[conda] torch 1.10.0+cu102 pypi_0 pypi
[conda] torch-geometric 2.0.2 pypi_0 pypi
[conda] torch-scatter 2.0.9 pypi_0 pypi
[conda] torch-sparse 0.6.12 pypi_0 pypi
[conda] torchaudio 0.10.0+cu102 pypi_0 pypi
[conda] torchmetrics 0.6.1 pypi_0 pypi
[conda] torchsummary 1.5.1 pypi_0 pypi
[conda] torchvision 0.11.1+cu102 pypi_0 pypi

cc @pmeier

[xianyuanliu]

This issue has been discussed in stackoverflow.

[pmeier]

Hey @xianyuanliu, thanks a lot for the report. I send a PR that fixes this, but it will only be included in the next release in ~3 months.

Until then, the answer on SO is spot on. I suggest you follow the first proposal and modify the files in-place.

import os
import pathlib

root = ...

for path in pathlib.Path(root).glob("*list*.txt"):
    with open(path, "r+") as file:
        content = [line.replace("/", os.sep) for line in file]
        file.seek(0)
        file.writelines(content)

Don't forget to make a backup of them to be able to restore them after the patch is released.

[xianyuanliu]

Many thanks! I will close this.