The labels in the QMNIST dataset are all equal to 0.

[Foobin]

🐛 Describe the bug

The labels in the QMNIST dataset are all equal to 0.

im_transform = T.Compose([
    T.ToTensor()
])

qmnist_train = torchvision.datasets.QMNIST('data/', train=True, download=True, transform=im_transform)

# the first image is a '5'
plt.imshow(qmnist_train[0][0][0])
# prints '0'
print(qmnist_train[0][1])

# the following never outputs anything
[print(qmnist_train[i][1]) for i in range(len(qmnist_train)) if qmnist_train[i][1] != 0]

Versions

Collecting environment information...
PyTorch version: 1.13.1
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-57-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.6.124
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 980 Ti
Nvidia driver version: 525.60.11
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] geotorch==0.3.0
[pip3] numpy==1.23.5
[pip3] torch==1.13.1
[pip3] torchaudio==0.13.1
[pip3] torchvision==0.14.1
[conda] blas                      1.0                         mkl  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] geotorch                  0.3.0                    pypi_0    pypi
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0           py310h7f8727e_0  
[conda] mkl_fft                   1.3.1           py310hd6ae3a3_0  
[conda] mkl_random                1.2.2           py310h00e6091_0  
[conda] numpy                     1.23.5          py310hd5efca6_0  
[conda] numpy-base                1.23.5          py310h8e6c178_0  
[conda] pytorch                   1.13.1          py3.10_cuda11.6_cudnn8.3.2_0    pytorch
[conda] pytorch-cuda              11.6                 h867d48c_1    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchaudio                0.13.1              py310_cu116    pytorch
[conda] torchvision               0.14.1              py310_cu116    pytorch

cc @pmeier

[pmeier]

Can reproduce.

[pmeier]

This is caused by a faulty byte flip in

vision/torchvision/datasets/mnist.py

Lines 525 to 528 in 7dc5e5b

	needs_byte_reversal = sys.byteorder == "little" and num_bytes_per_value > 1
	parsed = torch.frombuffer(bytearray(data), dtype=torch_type, offset=(4 * (nd + 1)))
	if needs_byte_reversal:
	parsed = parsed.flip(0)

which was introduced in #4598. Due to the extra num_bytes_per_value > 1 this can only trigger for QMNIST labels, which is why we didn't detect it before.

[pmeier]

Given the amount of discussion in #4882 and the final conclusion

vision/torchvision/prototype/utils/_internal.py

Lines 94 to 97 in 7dc5e5b

	# We cannot use torch.frombuffer() directly, since it only supports the native byte order of the system. Thus, we
	# read the data with np.frombuffer() with the correct byte order and convert it to the native one with the
	# successive .astype() call.
	return torch.from_numpy(np.frombuffer(buffer, dtype=np_dtype, count=count).astype(np_dtype[1:], copy=False))

I don't think byte flipping is possible in PyTorch. Tensor.flip just flips the elements along a dimension, but leaves the values as is:

>>> t = torch.arange(4)
>>> t
tensor([0, 1, 2, 3])
>>> t.flip(0)
tensor([3, 2, 1, 0])

I think we should revert #4598 unless something happened in the past year that allows us to do byte flipping from PyTorch easily. cc @NicolasHug

[pmeier]

Ok, byte flipping, i.e. changing the endianness, is possible with PyTorch, by viewing the data through the torch.uint8 dtype:

if needs_byte_reversal:
    parsed = (
        parsed.contiguous()
        .view(torch.uint8)
        .view(parsed.numel(), num_bytes_per_value)
        .flip(1)
        .flatten()
        .view(torch_type)
    )

Thanks for the tip @lezcano!

[NicolasHug]

Thanks a lot for investigating @pmeier, and @Foobin for the report.

Perf isn't critical at this stage of the dataset loading, so we can go with either solution (revert to np, or use the native pytorch code above).

Any pref Philip?

[pmeier]

Minimally leaning towards the PyTorch only solution, since we tried to phase out numpy as much as possible for our prototype datsets (see #4882). But I'm fine with either.

[lezcano]

I know you guys mentioned that perf is not an issue here, but the PyTorch version should be much faster on CPU due to @vfdev-5 's recent vectorisation of flip.

[NicolasHug]

Good to know! If we can confirm that with some quick benchmarks, we could also potentially simplify the code @pmeier mentioned above #7079 (comment) ?

[pmeier]

If we can confirm that with some quick benchmarks, we could also potentially simplify the code

I don't think we can. The code above was carefully designed to not make a memcopy unless needed. torch.flip always copies. So if we are after performance, I don't think we have much room for improvement.

[NicolasHug]

I meant "simplify" this code:

vision/torchvision/prototype/utils/_internal.py

Lines 94 to 97 in 7dc5e5b

	# We cannot use torch.frombuffer() directly, since it only supports the native byte order of the system. Thus, we
	# read the data with np.frombuffer() with the correct byte order and convert it to the native one with the
	# successive .astype() call.
	return torch.from_numpy(np.frombuffer(buffer, dtype=np_dtype, count=count).astype(np_dtype[1:], copy=False))

i.e. remove the numpy dependency and use the pytorch way