Conversion from bool to float sometimes produces tensor with values 255.0 for True instead of 1.0

[matwilso]

🐛 Bug

When I have two seemingly identical bool tensors with values: tensor([True, True, True, True]), one of them produces a float tensor with values: tensor([1.0, 1.0, 1.0, 1.0]). The other one produces a float tensor with values: tensor([255., 255., 255., 255.]).

To Reproduce

Steps to reproduce the behavior:

import numpy as np
import torch
# these are the same data and they look the same when you load them, but they behave subtly differently
poisoned = np.frombuffer(b'\xff\xff\xff\xff', dtype=np.bool) # hex 255
clean = np.frombuffer(b'\x01\x01\x01\x01', dtype=np.bool)

torch_poisoned = torch.from_numpy(poisoned)
torch_clean = torch.from_numpy(clean)


print('poisoned == clean -->', (poisoned == clean).all())  # >>> True
print('torch_poisoned == torch_clean --> ', (torch_poisoned == torch_clean).all())  # >>> tensor(False)

print('torch_poisoned.dtype == torch_clean.dtype --> ', (torch_poisoned.dtype == torch_clean.dtype))  # >>> True
print('torch_poisoned.dtype, torch_clean.dtype --> ', torch_poisoned.dtype, torch_clean.dtype)  # >>> torch.bool, torch.bool

print('torch_clean: ',torch_clean)  # >>> tensor([True, True, True, True])
print('torch_clean.float(): ', torch_clean.float())  # >>> tensor([1., 1., 1., 1.])

print('torch_poisoned: ', torch_poisoned)  # >>> tensor([True, True, True, True])
print('torch_poisoned.float(): ', torch_poisoned.float())  # >>> tensor([255., 255., 255., 255.])

Expected behavior

Any thing that torch calls a True bool should be converted to a float value of 1.0, not whatever bytes happened to be stored inside the bool.

This could just be an abuse of bool, but I came across this bug when using converting a boolean image using PIL. PyTorch should probably prevent users from being able to shoot their foot like this.

Environment

Collecting environment information...
PyTorch version: 1.7.1+cu110
Is debug build: False
CUDA used to build PyTorch: 11.0
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.15.3

Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 11.2.142
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 455.45.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.1.1
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.7.1+cu110
[pip3] torchaudio==0.7.2
[pip3] torchgeometry==0.1.2
[pip3] torchvision==0.8.2+cu110
[conda] Could not collect

Additional context

cc @ezyang @gchanan @zou3519 @mruberry @rgommers @heitorschueroff

[mruberry]

NumPy will handle this case as expected:

poisoned.astype(np.float)
: array([1., 1., 1., 1.])

I think we would accept a PR fixing this as long as there wasn't a performance penalty.

[relh]

this also happens if you call .int()

[relh]

Also, a related issue, when on a GPU:

(Pdb) torch_poisoned.to(0).max()
tensor(False, device='cuda:0')

(Pdb) torch_clean.to(0).max()
tensor(True, device='cuda:0')

When not on a GPU:

(Pdb) torch_poisoned.max()
tensor(True)

(Pdb) torch_clean.max()
tensor(True)

[relh]

This also impacts doing arithmetic with these poisoned tensors.

These poisoned tensors exist in a fairly common use case. If you load PNG binary masks into boolean tensors and move them to GPUs they will act like the poisoned tensors above.

This happens even if you call np.array rather than np.asarray on the PIL Image

[jeromerony]

I can confirm that this also happens on 1.8.2 LTS.

It also happens when converting a PIL binary image to tensor.

import numpy as np
import torch
from PIL import Image, ImageDraw

mask = Image.new(mode='1', size=(4, 4), color=False)
draw = ImageDraw.Draw(mask)
draw.polygon([(1, 1), (2, 1), (2, 2), (1, 2)], fill=True)

np_mask = np.array(mask)
print(np_mask)
tensor_mask = torch.from_numpy(np_mask)
print(tensor_mask) 
float_mask = tensor_mask.float()
print(float_mask)

[jeromerony]

It's also important to note that the behavior differs based on the device. On CPU, True values will be converted to 255 while on a CUDA device, True will be converted to -1.

import numpy as np
import torch

device = torch.device('cuda')
a = np.frombuffer(b'\xff\x00', dtype=bool)
print(a)                # >>> [ True False]
t = torch.from_numpy(a).to(device)
print(t)                # >>> tensor([ True, False], device='cuda:0')
print(t.float())        # >>> tensor([-1.,  0.], device='cuda:0')

[rfeinman]

I'm having this same issue, and I get different results on Linux vs. MacOS conda builds (both v1.10.1). It would be great to sort this out to foster reproducibility across different environments. It's especially important when using binary PIL images (mode='1'); in these cases np.array(im) yields a bool array with "True" bytes of value \xff).

Code snippet:

a = np.frombuffer(b'\xff\x00', dtype=np.bool_)
t = torch.tensor(a, dtype=torch.float)
print(t)

Linux output:

tensor([255.,   0.])

MacOS output:

tensor([1., 0.])

@mruberry any idea why the results differ between linux and macos distributions?

[mruberry]

@rfeinman I don't immediately no, sorry. I had more context on this issue previously but I've forgotten most of it. My (very) vague recollection is that the C++ standard doesn't require the boolean True value be represented by 1 but any non-zero value, so different implementations may be doing different things while still being compliant. Still, even though the C++ standard may not guarantee this behavior it's clearly surprising to PyTorch users. Bumping priority based on user activity.