Python float to torch.float32

[whatdhack]

🐛 Describe the bug

Noticing the following . As I understand, python float is 64 bit. Hence , converting to torch.float32 would be lossy. What is the recommended way to preserve as much accuracy as possible in this transition ?

python -c "import torch;import time;offset=1726274430;torch.set_printoptions(precision=10);l1=[time.time()-offset,time.time()-offset];print('list:', l1, type(l1[0]));t1=torch.tensor(l1);print('cpu:', t1, t1.dtype);t1=t1.to('cuda'); print('gpu:', t1, t1.dtype);"

list: [521468.6812365055, 521468.6812376976] <class 'float'>
cpu: tensor([521468.6875000000, 521468.6875000000]) torch.float32
gpu: tensor([521468.6875000000, 521468.6875000000], device='cuda:0') torch.float32 

Versions

N/A

[kshitij12345]

I think you can just manually specify the dtype to be torch.double.

import torch
import time
offset=1726274430
torch.set_printoptions(precision=10)
l1=[time.time()-offset,time.time()-offset]
print('list:', l1, type(l1[0]))
t1=torch.tensor(l1, dtype=torch.double)
print('cpu:', t1, t1.dtype)
t1=t1.to('cuda')
print('gpu:', t1, t1.dtype)

prints

list: [565403.2435054779, 565403.2435059547] <class 'float'>
cpu: tensor([565403.2435054779, 565403.2435059547], dtype=torch.float64) torch.float64
gpu: tensor([565403.2435054779, 565403.2435059547], device='cuda:0',
       dtype=torch.float64) torch.float64

[whatdhack]

Thanks. The constraint is torch.float32 for follow on processing. In that situation, is there any recommended to preserve accuracy ? Would converting to numpy.floa32 before tensorization preserve more accuracy ?

[malfet]

Please use https://discuss.pytorch.org/ to ask generic questions on how to use PyTorch and issue to report problems/suggest new features.

As @kshitij12345 correctly replied, you can create float64 tensor if you choose to.

Would converting to numpy.floa32 before tensorization preserve more accuracy ?

No, to the best of my knowledge it should not, but if you find otherwise, please do not hesitate to create an issue to report something like that.