Failure when converting a sparse boolean tensor to a dense tensor

[NathanHowell]

🐛 Bug

Converting a sparse boolean tensor to a dense tensor fails

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-4-183b7da37f3a> in <module>
----> 1 torch.tensor([True, False]).to_sparse().to_dense()

RuntimeError: "add_dense_sparse" not implemented for 'Bool'

To Reproduce

Steps to reproduce the behavior:

import torch

# raises exception
torch.tensor([True, False]).to_sparse().to_dense()
# RuntimeError

# converting via another type works
torch.tensor([True, False]).to_sparse().to(torch.uint8).to_dense().to(torch.bool)
# Out[6]: tensor([ True, False])

Expected behavior

Conversion to succeed

Environment

PyTorch version: 1.7.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.1 (x86_64)
GCC version: Could not collect
Clang version: 12.0.0 (clang-1200.0.32.28)
CMake version: version 3.19.2

Python version: 3.9 (64-bit runtime)
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.4
[pip3] torch==1.7.1
[pip3] torchvision==0.8.2
[conda] Could not collect

cc @izdeby @aocsa @nikitaved @pearu @mruberry

[mruberry]

Thank you for reporting this issue, @NathanHowell. We would accept a PR implementing this behavior.

[NathanHowell]

Thank you for reporting this issue, @NathanHowell. We would accept a PR implementing this behavior.

do you have any pointers as to how this can be fixed? I haven't run this through gdb/lldb and quick scan through the aten code didn't show anything obvious.

[mruberry]

I would suggest the following:

First, understand the call chain on CPU and CUDA. In this case the error message suggests the issue is from here on CPU:

pytorch/aten/src/ATen/native/sparse/SparseTensorMath.cpp

Line 653 in 92f37ae

AT_DISPATCH_ALL_TYPES(

And if we look at that dispatch macro we can see it doesn't include bool:

pytorch/aten/src/ATen/Dispatch.h

Line 404 in f10b53d

#define AT_DISPATCH_ALL_TYPES(TYPE, NAME, ...) \

This suggests a good starting point for inserting debug statements and prints, for example, on both the CPU and in the analogous place on CUDA.

Second, this fix might be as simple as updating the dispatch macro to include the boolean dtype. So you could try that and validate it fixes the issue, then create a test in test_sparse.py to verify the behavior works as expected. If that doesn't work, however, you'll have to dig further into the CPU and CUDA code to understand what's required to support this operation for bools.

[NathanHowell]

cool, thx. I'll take a peek in the next day or two

[KinWaiCheuk]

I still encounter this bug after using torch.save and torch.load.

This code works:

x = torch.tensor([[[True, False],[True, True],[True, True]]])
x_sparse = x.to_sparse()
x_dense = x_sparse.to_dense()

But after doing torch.save and torch.load, I get the following error

torch.save(x_sparse, 'x.pt') 
x_loaded = torch.load('x.pt')
x_loaded.to_dense()

>>> RuntimeError: "coalesce" not implemented for 'Bool'