Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch.flatten() returns a 0-dim (not 1-dim) tensor for 0-dim tensors #22963

Closed
szali opened this issue Jul 17, 2019 · 8 comments

Comments

@szali
Copy link

commented Jul 17, 2019

馃悰 Bug

torch.flatten works in a counterintuitive way when supplied with 0-dim tensors.

In numpy:

np.asfarray(123).flatten()
array([123.]) # returns a 1-dim array
np.asfarray([1,2,3]).flatten()
array([1., 2., 3.]) # returns a 1-dim array

So, in numpy, flatten() always returns a 1-dim array, which is exactly why one would use it.
In contrast, in pytorch, it returns a 0-dim tensor for 0-dim tensors, which defeats the whole purpose of flatten: to convert all tensors to 1-dim, so we can handle arbitrarily shaped tensors in a uniform way.

In torch:

torch.tensor(123).flatten()
tensor(123) # returns a 0-dim tensor!
torch.tensor([1,2,3]).flatten()
tensor([1, 2, 3]) # returns a 1-dim tensor

To Reproduce

Steps to reproduce the behavior:

  1. Call .flatten() or torch.flatten() on a 0-dim tensor

Expected behavior

The tensor becomes 1-dimensional.

Environment

PyTorch version: 1.2.0a0+c9f41e9
Is debug build: No
CUDA used to build PyTorch: 10.1.105

OS: Ubuntu 16.04.3 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
CMake version: version 3.14.5

Python version: 3.5
Is CUDA available: Yes
CUDA runtime version: 10.1.105
GPU models and configuration:
GPU 0: TITAN X (Pascal)
GPU 1: TITAN X (Pascal)

Nvidia driver version: 418.56
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.5.1
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a

Versions of relevant libraries:
[pip3] numpy (1.16.4)
[pip3] torch (1.2.0a0+c9f41e9)
[pip3] torchfile (0.1.0)
[pip3] torchvision (0.1.9)
[conda] Could not collect

cc @ezyang @SsnL

@fmassa

This comment has been minimized.

Copy link
Member

commented Jul 17, 2019

PyTorch implementation of flatten is not equivalent to numpy's, because we can specify start_dim and end_dim. This means that PyTorch flatten, by definition, doesn't need to return a 1d tensor.

But it might be worth discussing if we want to change the behavior for this corner case, in order to be more aligned with numpy. I'm not sure if it would make the semantics of our current operator weird / complicated though.

@SsnL

This comment has been minimized.

Copy link
Collaborator

commented Jul 17, 2019

I agree with @fmassa . PyTorch and np's flatten are a bit different. The PyTorch equivalent of np.flatten is more like reshape(-1).

@szali

This comment has been minimized.

Copy link
Author

commented Jul 17, 2019

I think many people using PyTorch are coming from a numpy background and they expect torch.flatten without extra arguments to behave like np.flatten. I understand that the version with the range arguments cannot work the same way as np.flatten for 0d tensors, because it can only reduce the number of dimensions, but I don't see why that would prevent the argumentless version from mimicking the behaviour of np.flatten. Personally I think that would result in less confusion for most users.

@SsnL

This comment has been minimized.

Copy link
Collaborator

commented Jul 17, 2019

I'm not saying that we shouldn't change. I'm just pointing out the difference in two methods of these two libraries. :)

@jeffreyksmithjr

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

I think this also gets into the questions from @umanwizard 's project around making PT more like numpy. One of the ideas kicked around in that project was to build a compatibility layer, but it sounds like this discussion is more about people who may not even realize that PT and numpy have subtly different behaviors, so they wouldn't know that they needed to "opt-in" and use a compatibility feature. Agreeing that this needs discussion.

@rgommers

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

@cpuhrsch you removed "needs discussion", but what was the outcome? Change to numpy's semantics or leave as is?

@gchanan

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

personal opinion: I think it should return a 1-dimensional tensor. It's essentially turning the range of dimensions (start_dim to end_dim) into a 1-dimensional tensor, and then stacking the dimensions before start_dim and after end_dim onto it.

@izdeby

This comment has been minimized.

Copy link
Contributor

commented Jul 29, 2019

Resolution: we should return 1-dim tensor

CamiWilliams added a commit that referenced this issue Aug 30, 2019
zdevito pushed a commit to zdevito/ATen that referenced this issue Aug 30, 2019
Summary:
PR for `torch.flatten()` to return a 1-dim tensor on a 0-dim tensor

> torch.tensor(123).shape -> torch.Size([])
> torch.tensor(123).flatten() -> torch.tensor([123])
> torch.tensor(123).flatten().shape -> torch.Size([1])

resolve pytorch/pytorch#22963
Pull Request resolved: pytorch/pytorch#25406

Differential Revision: D17120464

Pulled By: CamiWilliams

fbshipit-source-id: efbecd61f0aefd82f2ab417ca6bb467488ff99de
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can鈥檛 perform that action at this time.