Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dimension out of range (expected to be in range of [-1, 0], but got 1) #5554

Closed
pranavkanade opened this issue Mar 4, 2018 · 27 comments
Closed

dimension out of range (expected to be in range of [-1, 0], but got 1) #5554

pranavkanade opened this issue Mar 4, 2018 · 27 comments
Assignees
@ailzhang

Comments

@pranavkanade
Copy link 

criterion = nn.CrossEntropyLoss()

print(outputs.data)
print(label.data)
loss = criterion(outputs, label)      # getting error at this point

The output that I'm getting is =>

 0.0174
 0.1866
[torch.FloatTensor of size 2]


 0
 1
[torch.FloatTensor of size 2]

This is correct as far as the documentation is concern, but still getting following error
dimension out of range (expected to be in range of [-1, 0], but got 1)
@zou3519
Copy link
Contributor

zou3519 commented Mar 5, 2018

From the docs:
image
I'm assuming your C = 2 and N = 2. Then target should be a tensor of size (2,) while output should be a tensor of size (2, 2).

@apaszke
Copy link
Contributor

apaszke commented Mar 5, 2018

@zou3519 is right, but let’s keep this issue open until we fix the error message

@pranavkanade
Copy link
Author

Thanks @zou3519,
That means, I need to have the dimension of the outputs tensor (1,2) and label tensor (1) as I am evaluating single example at a time, right ??

@zou3519
Copy link
Contributor

zou3519 commented Mar 27, 2018

@pskanade you have two labels, yes? Then your output should be of size (2, 2), where outputs[0] contains two elements that are the probability that outputs[0] is in either class. Same for outputs[1].

@zou3519 zou3519 mentioned this issue Mar 28, 2018
Merged
zou3519 added a commit to zou3519/pytorch that referenced this issue Mar 28, 2018
@zou3519
NLLLoss: error message for mismatched input/target batch sizes
336e101 
Fixes pytorch#5554

Adds an error message for when NLLLoss is passed an input and target
whose batch sizes don't match. Ideally this check should live in ATen
but since there is NLLLoss logic in python the check is there right now.
soumith pushed a commit that referenced this issue Mar 28, 2018
@zou3519 @soumith
NLLLoss: error message for mismatched input/target batch sizes (#6072)
371e14b 
Fixes #5554

Adds an error message for when NLLLoss is passed an input and target
whose batch sizes don't match. Ideally this check should live in ATen
but since there is NLLLoss logic in python the check is there right now.
@mithunpaul08
Copy link

mithunpaul08 commented May 15, 2018 •
edited

Just curious: Is this change pushed in the latest pyTorch version? I seem to be getting the same error message. Am on pyTorch 0.4.0

@zou3519
Copy link
Contributor

zou3519 commented May 16, 2018

@mithunpaul08 this should be in 0.4. Could you post a code sample that gives the error message?

@mithunpaul08
Copy link

@zou3519 pasted below is the code. I am also printing the size of the tensors below. But I get error in the loss line.
Code:

loss_fn = nn.CrossEntropyLoss()
pred_y = model(featureV)
label_vector_np=label_vector.astype(np.float64)
label_vector_np_tensor_v=convert_numpy_tensor_variable(label_vector_np)
loss_training = loss_fn(pred_y, label_vector_np_tensor_v)

Error:

File ~/utils/ffnn.py", line 205, in train_using_ffnn 
loss_training = loss_fn(pred_y, label_vector_np_tensor_v)
RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)

Sizes of tensors:

len(pred_y):
torch.Size([3])
len(label_vector_np_tensor_v):
torch.Size([1, 3])

@zou3519
Copy link
Contributor

zou3519 commented May 16, 2018

The input to NLLLoss should be of size (N, C), while the target is of size (C,).
The way one calls NLLLoss is loss_fn(input, target). Right now your input and target tensors don't have the right size.

That being said, I can't reproduce your error message on 0.4. This it what happens when I use tensors of the same size as your tensors:


In [1]: import torch
   ...: import torch.nn as nn
   ...:
   ...: loss_fn = nn.NLLLoss()
   ...: input = torch.randn(3)
   ...: target = torch.ones(1, 3, dtype=torch.long)
   ...: loss_fn(input, target)
   ...:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-bdb29da7c6a9> in <module>()
      5 input = torch.randn(3)
      6 target = torch.ones(1, 3, dtype=torch.long)
----> 7 loss_fn(input, target)

~/pytorch/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    489             result = self._slow_forward(*input, **kwargs)
    490         else:
--> 491             result = self.forward(*input, **kwargs)
    492         for hook in self._forward_hooks.values():
    493             hook_result = hook(self, input, result)

~/pytorch/torch/nn/modules/loss.py in forward(self, input, target)
    191         _assert_no_grad(target)
    192         return F.nll_loss(input, target, self.weight, self.size_average,
--> 193                           self.ignore_index, self.reduce)
    194
    195

~/pytorch/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce)
   1336     dim = input.dim()
   1337     if dim < 2:
-> 1338         raise ValueError('Expected 2 or more dimensions (got {})'.format(dim))
   1339
   1340     if input.size(0) != target.size(0):

ValueError: Expected 2 or more dimensions (got 1)

@mithunpaul08
Copy link

mithunpaul08 commented May 16, 2018 •
edited

ok, i figured it out. So the key point which i didn't understand from the documentation was that the TARGET should be just one entry saying which class it belongs to Eg:[2] instead of a one hot vector like [0,0,1]. I mean, frankly, I imagined the input and target being of similar shape (which is more intuitive). i.e Now that the input is a vector of [0,0,1] i imagined the TARGET also should be in same shape. Anyway, am glad it worked out. I would love to have a better worded documentation though, imho.

Below is the code/shapes without dimension error.

len(pred_y):
torch.Size([1, 3])
tensor([[ 0.0000,  0.0000,  0.1527]])
len(x):
torch.Size([1])
tensor([ 2])
loss_training = loss_fn(pred_y, x)

@zou3519
Copy link
Contributor

zou3519 commented May 16, 2018

@mithunpaul08 please feel free to send a PR to improve the wording of the documentation!

@mithunpaul08
Copy link

Pretty stupid but, was trying to edit documentation but couldn't find it. Isn't the github page for the documentation of nn.html (which inturn contains torch.nn.CrossEntropyLoss) kept here

Wasn't able to understand the layout. Will you kindly be able to point me to the right page so that I can edit it and submit a pull request? If its too much pain, ignore please.

@zou3519
Copy link
Contributor

zou3519 commented May 16, 2018

If you edit the docstring for nn.NLLLoss here, the changes will be reflected at our documentation website.

You can preview a copy on your computer by cd-ing into pytorch/docs and running make html.

mithunpaul08 added a commit to mithunpaul08/pytorch that referenced this issue Jun 1, 2018
@mithunpaul08
added examples and more wordings to documentation of cross entropy lo…
5ae2139 
…ss as sugested by developers in issue pytorch#5554
@mithunpaul08 mithunpaul08 mentioned this issue Jun 1, 2018
Closed
@mithunpaul08
Copy link

mithunpaul08 commented Jun 1, 2018 •
edited

@zou3519 have created a pull request. pending travis checks. Please approve if it looks ok. Sorry it took this long.

@horsepurve horsepurve mentioned this issue Oct 30, 2018
Open
@Totoro-wen
Copy link

Totoro-wen commented Apr 1, 2019 •
edited

Hello,i met this same porblem in pytorch1.0.1.

outputs = model(img)            
predicted_source, predicted = torch.max(outputs.data, 1)
loss = criterion(predicted.double(), labels.double())

ValueError: Expected target size (1, 3), got torch.Size([1])
Thanks for your help.

@DineshShrestha
Copy link

IndexError Traceback (most recent call last)
in
1 im_id = '23445819_3a458716c1.jpg'
----> 2 sample_and_plot(im_id)

in sample_and_plot(im_id)
35 s1 = solver.sample(feature, search_mode='greedy')
36 s1 = decode_captions(s1, data.idx_to_word)
---> 37 s2 = solver.sample(feature, search_mode='beam', b_size=5)
38 s2 = decode_captions(s2, data.idx_to_word)
39 s3 = solver.sample(feature, search_mode='beam', b_size=10)

~\OneDrive\Desktop\image_caption_pytorch\solver.py in sample(self, features, max_length, b_size, model_mode, search_mode)
232 if t == 0:
233 scores = scores.log() # (N, V)
--> 234 sorted_scores, idxes = scores.topk(b_size, dim=1) # (N, B)
235 candidates = idxes.unsqueeze(-1) # (N, B, 1)
236 feeds = torch.reshape(candidates, (-1,1)) # (N*B, 1)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

@hayatkhan8660-maker
Copy link

hayatkhan8660-maker commented Oct 14, 2020 •
edited

Hi @mithunpaul08 , i am working on a project related to person re-identification. I am trying to re-implement that code of one of the CVPR paper entitled "ABD-Net: Attentive but Diverse Person Re-Identification". I trained the ABD-Net architecture with resnet and densenet but when i am trying to train the architecture using shufflenet backbone I face this error. Could you please help me..

=================================================================
File "train.py", line 147, in main
train(epoch, model, criterion, regularizer, optimizer, trainloader, use_gpu, fixbase=True)
File "train.py", line 246, in train
loss = criterion(outputs, pids)
File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "F:\hayat ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 56, in forward
return self._forward(inputs[1], targets)
File "F:\hayatullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 52, in _forward
return sum([self.apply_loss(x, targets) for x in inputs_tuple]) / len(inputs_tuple)
File "F:\hayat ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 52, in
return sum([self.apply_loss(x, targets) for x in inputs_tuple]) / len(inputs_tuple)
File "F:\hayat ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 32, in apply_loss
log_probs = self.logsoftmax(inputs)
File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\activation.py", line 1179, in forward
return F.log_softmax(input, self.dim, _stacklevel=5)
File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\functional.py", line 1350, in log_softmax
ret = input.log_softmax(dim)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

@DineshShrestha
Copy link

DineshShrestha commented Oct 14, 2020 via email

@hayatkhan8660-maker
Copy link

Just change the image file and the text data so that the number of images and text corresponds to each other. Dinesh
…
On Wed, 14 Oct 2020, 06:26 Hayat ullah, @.***> wrote: Hi @mithunpaul08 https://github.com/mithunpaul08 , i am working on project related to person re-identification i am trying to re-implement that code of one of the CVPR paper entitled "ABD-Net: Attentive but Diverse Person Re-Identification". I trained the ABD-Net architecture with resnet and densenet but when i am trying to train the architecture using shufflenet backbone I face this error. Could you please help me.. ================================================================= File "train.py", line 147, in main train(epoch, model, criterion, regularizer, optimizer, trainloader, use_gpu, fixbase=True) File "train.py", line 246, in train loss = criterion(outputs, pids) File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "F:\Sami ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 56, in forward return self._forward(inputs[1], targets) File "F:\Sami ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 52, in _forward return sum([self.apply_loss(x, targets) for x in inputs_tuple]) / len(inputs_tuple) File "F:\Sami ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 52, in return sum([self.apply_loss(x, targets) for x in inputs_tuple]) / len(inputs_tuple) File "F:\Sami ullah work\Attention Code\paper 2_new code\torchreid\losses\cross_entropy_loss.py", line 32, in apply_loss log_probs = self.logsoftmax(inputs) File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\modules\activation.py", line 1179, in forward return F.log_softmax(input, self.dim, _stacklevel=5) File "C:\Users\Hayat\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\nn\functional.py", line 1350, in log_softmax ret = input.log_softmax(dim) IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5554 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTKXHO43MAH25YJI3LUGFDSKUSBBANCNFSM4ETNTY2A .

Could you please explain in a bit detail.. thank you

@DineshShrestha
Copy link

DineshShrestha commented Oct 14, 2020 via email

@bhav09
Copy link

bhav09 commented Dec 25, 2020

Class Dataset

class dataset(Dataset):
    def __init__(self):
        self.tf=TfidfVectorizer(max_df=0.99, min_df=0.005)
        self.x=self.tf.fit_transform(corpus).toarray()
        self.y=list(df.review)
        self.x_train,self.x_test,self.y_train,self.y_test=train_test_split(self.x,self.y,test_size=0.2)
        self.token2idx=self.tf.vocabulary_
        self.idx2token = {idx: token for token, idx in self.token2idx.items()}
        print(self.idx2token)
    
    def __getitem__(self,i):
        return self.x_train[i, :], self.y_train[i]
    
    def __len__(self):
        return self.x_train.shape[0]

Class Classifier

class classifier(nn.Module):
    def __init__(self,vocab_size,hidden1,hidden2):
        super(classifier,self).__init__()
        self.fc1=nn.Linear(vocab_size,hidden1)
        self.fc2=nn.Linear(hidden1,hidden2)
        self.fc3=nn.Linear(hidden2,1)
    def forward(self,inputs):
        x=F.relu(self.fc1(inputs.squeeze(1).float()))
        x=F.relu(self.fc2(x))
        return self.fc3(x)

Training Loop

epochs=10
total=0
model.train()
for epoch in tqdm(range(epochs)):
    progress_bar=tqdm_notebook(train_loader,leave=False)
    losses=[]
    correct=0
    for inputs,target in progress_bar:
        model.zero_grad()
        output=model(inputs)
        print(output.squeeze().shape)
        print(target.shape)
        loss=criterion(output.squeeze(),target.float())
        loss.backward()
        nn.utils.clip_grad_norm_(model.parameters(), 3)
        optim.step()
        correct += (output == target).float().sum()
        progress_bar.set_description(f'Loss: {loss.item():.3f}')
        losses.append(loss.item())
        total += 1
    epoch_loss = sum(losses) / total
    train_losses.append(epoch_loss)   
    tqdm.write(f'Epoch #{epoch + 1}\tTrain Loss: {epoch_loss:.3f}\tAccuracy: {correct/output.shape[0]}')

Error

IndexError                                Traceback (most recent call last)
<ipython-input-78-6b86c97bcabf> in <module>
     14         print(output.squeeze().shape)
     15         print(target.shape)
---> 16         loss=criterion(output.squeeze(),target.float())
     17         loss.backward()
     18         nn.utils.clip_grad_norm_(model.parameters(), 3)

~\Anaconda3\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
    930     def forward(self, input, target):
    931         return F.cross_entropy(input, target, weight=self.weight,
--> 932                                ignore_index=self.ignore_index, reduction=self.reduction)
    933 
    934 

~\Anaconda3\lib\site-packages\torch\nn\functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2315     if size_average is not None or reduce is not None:
   2316         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2317     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2318 
   2319 

~\Anaconda3\lib\site-packages\torch\nn\functional.py in log_softmax(input, dim, _stacklevel, dtype)
   1533         dim = _get_softmax_dim('log_softmax', input.dim(), _stacklevel)
   1534     if dtype is None:
-> 1535         ret = input.log_softmax(dim)
   1536     else:
   1537         ret = input.log_softmax(dim, dtype=dtype)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

@DineshShrestha
Copy link

DineshShrestha commented Dec 25, 2020 via email

@kaelzhang
Copy link

Just curious: Is this change pushed in the latest pyTorch version? I seem to be getting the same error message. Am on pyTorch 0.4.0

Same error message for torch 1.8.0

I think the issue still exists after so many years.

@DineshShrestha
Copy link

DineshShrestha commented Mar 13, 2021 via email

@zou3519
Copy link
Contributor

zou3519 commented Mar 15, 2021

@kaelzhang @DineshShrestha please ask in the forums https://discuss.pytorch.org/ (or open an issue if you think this is a bug). It's hard for us to keep track of activity on closed issues especially if they've been closed for a while.

@zou3519
Copy link
Contributor

zou3519 commented Jun 25, 2021

@m416kar98k please open a new issue.

@abbaskali abbaskali mentioned this issue Mar 7, 2022
Open
@sumorday
Copy link

output1.view(1, -1)

try this

@fionavictoria
Copy link

fionavictoria commented Aug 5, 2022 •
edited

If "outputs = model(ids, mask, token_type_ids).squeeze()"
causes [dimension out of range (expected to be in range of [-1, 0], but got 1)] when calculating the loss.

The error can be rectified by removing the squeeze function. [ outputs = model(ids, mask, token_type_ids) ] works fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ailzhang ailzhang

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests