torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet

[Zhao-Yuting]

when i extracted the features and boxes, i received the error message following. would you please help me to find the reason? thanks a lot.

error message:
raise NotSupportedError(base.range(), "slicing multiple dimensions at the same time isn't supported yet")
torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet
proposals (Tensor): boxes to be encoded
"""

# perform some unpacking to make it JIT-fusion friendly
wx = weights[0]
wy = weights[1]
ww = weights[2]
wh = weights[3]

proposals_x1 = proposals[:, 0].unsqueeze(1)
               ~~~~~~~~~ <--- HERE
proposals_y1 = proposals[:, 1].unsqueeze(1)
proposals_x2 = proposals[:, 2].unsqueeze(1)
proposals_y2 = proposals[:, 3].unsqueeze(1)

reference_boxes_x1 = reference_boxes[:, 0].unsqueeze(1)
reference_boxes_y1 = reference_boxes[:, 1].unsqueeze(1)
reference_boxes_x2 = reference_boxes[:, 2].unsqueeze(1)
reference_boxes_y2 = reference_boxes[:, 3].unsqueeze(1)

[violetteshev]

It seems that proposals tensor has more than 2 dimensions, so it raises an exception. Can you post the whole code where you get this error?

[Zhao-Yuting]

thank you in advance.
after finishing the Installation step by step, I run the feature extraction command line as follows:
"python extract_features.py --image_dir /home/yuting/multi30k/clone/image/test_2017_flickr --out_dir features"

the error occured:
Traceback (most recent call last):
File "extract_features.py", line 10, in
from model.faster_rcnn.resnet import resnet
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/resnet.py", line 6, in
from model.faster_rcnn.faster_rcnn import _fasterRCNN
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/faster_rcnn.py", line 6, in
import torchvision.models as models
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/init.py", line 1, in
from torchvision import models
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/models/init.py", line 12, in
from . import detection
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/models/detection/init.py", line 1, in <module

from .faster_rcnn import *

File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/models/detection/faster_rcnn.py", line 13, in
from .rpn import AnchorGenerator, RPNHead, RegionProposalNetwork
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/models/detection/rpn.py", line 8, in
from . import _utils as det_utils
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torchvision/models/detection/_utils.py", line 74, in
@torch.jit.script
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/init.py", line 364, in script
graph = _script_graph(fn, _frames_up=_frames_up + 1)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/init.py", line 359, in _script_graph
ast = get_jit_ast(fn)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 132, in get_jit_ast
return build_def(SourceRangeFactory(source), py_ast.body[0])
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 151, in build_def
build_stmts(ctx, body))
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 123, in build_stmts
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 123, in
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 205, in build_Assign
rhs = build_expr(ctx, stmt.value)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 314, in build_Call
func = build_expr(ctx, expr.func)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 300, in build_Attribute
value = build_expr(ctx, expr.value)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/home/yuting/.pyenv/versions/env/lib/python3.6/site-packages/torch/jit/frontend.py", line 422, in build_Subscript
raise NotSupportedError(base.range(), "slicing multiple dimensions at the same time isn't supported yet")
torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet
proposals (Tensor): boxes to be encoded
"""

# perform some unpacking to make it JIT-fusion friendly
wx = weights[0]
wy = weights[1]
ww = weights[2]
wh = weights[3]

proposals_x1 = proposals[:, 0].unsqueeze(1)
               ~~~~~~~~~ <--- HERE
proposals_y1 = proposals[:, 1].unsqueeze(1)
proposals_x2 = proposals[:, 2].unsqueeze(1)
proposals_y2 = proposals[:, 3].unsqueeze(1)

reference_boxes_x1 = reference_boxes[:, 0].unsqueeze(1)
reference_boxes_y1 = reference_boxes[:, 1].unsqueeze(1)
reference_boxes_x2 = reference_boxes[:, 2].unsqueeze(1)
reference_boxes_y2 = reference_boxes[:, 3].unsqueeze(1)

[violetteshev]

So the problem is with torch (or torchvision) version. I used the following packages all installed with pip:

cffi==1.12.2  
cycler==0.10.0  
Cython==0.29.5  
easydict==1.9     
kiwisolver==1.0.1   
matplotlib==3.0.2   
msgpack==0.6.1   
numpy==1.16.2  
opencv-python==4.0.0.21
Pillow==5.4.1   
pip==19.0.3  
protobuf==3.6.1   
pycparser==2.19    
pyparsing==2.3.1   
python-dateutil==2.8.0   
PyYAML==3.13    
scipy==1.2.1   
setuptools==40.8.0  
six==1.12.0  
tensorboardX==1.6     
torch==0.4.0   
torchvision==0.2.1
tqdm==4.31.1  
wheel==0.33.1

Please try once again with these versions.

[Zhao-Yuting]

thank you for your quick help! I will have a try soon and have a look.

[Zhao-Yuting]

thank you in advance.
I have installed packages as you instructed above. another error occurred as follows:

Traceback (most recent call last):
File "extract_features.py", line 10, in
from model.faster_rcnn.resnet import resnet
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/resnet.py", line 6, in
from model.faster_rcnn.faster_rcnn import _fasterRCNN
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/faster_rcnn.py", line 10, in
from model.rpn.rpn import _RPN
File "/home/yuting/bottom-up-features/lib/model/rpn/rpn.py", line 10, in
from model.utils.net_utils import _smooth_l1_loss
File "/home/yuting/bottom-up-features/lib/model/utils/net_utils.py", line 8, in
from model.roi_crop.functions.roi_crop import RoICropFunction
File "/home/yuting/bottom-up-features/lib/model/roi_crop/functions/roi_crop.py", line 4, in
from .._ext import roi_crop
File "/home/yuting/bottom-up-features/lib/model/roi_crop/_ext/roi_crop/init.py", line 3, in
from ._roi_crop import lib as _lib, ffi as _ffi
ImportError: /home/yuting/bottom-up-features/lib/model/roi_crop/_ext/roi_crop/_roi_crop.so: undefined symbol: __cudaPopCallConfiguration

[violetteshev]

Did you recompile the library after reinstalling packages?

[Zhao-Yuting]

yes. after installing packages, I do the command line "cd lib" and "sh make.sh" to compile the code. then run the extraction command "python extract_features.py --image_dir /home/yuting/multi30k/clone/image/test_2017_flickr --out_dir features"

the error message is also as following:
Traceback (most recent call last):
File "extract_features.py", line 10, in
from model.faster_rcnn.resnet import resnet
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/resnet.py", line 6, in
from model.faster_rcnn.faster_rcnn import _fasterRCNN
File "/home/yuting/bottom-up-features/lib/model/faster_rcnn/faster_rcnn.py", line 10, in
from model.rpn.rpn import _RPN
File "/home/yuting/bottom-up-features/lib/model/rpn/rpn.py", line 10, in
from model.utils.net_utils import _smooth_l1_loss
File "/home/yuting/bottom-up-features/lib/model/utils/net_utils.py", line 8, in
from model.roi_crop.functions.roi_crop import RoICropFunction
File "/home/yuting/bottom-up-features/lib/model/roi_crop/functions/roi_crop.py", line 4, in
from .._ext import roi_crop
File "/home/yuting/bottom-up-features/lib/model/roi_crop/_ext/roi_crop/init.py", line 3, in
from ._roi_crop import lib as _lib, ffi as _ffi
ImportError: /home/yuting/bottom-up-features/lib/model/roi_crop/_ext/roi_crop/_roi_crop.so: undefined symbol: __cudaPopCallConfiguration

[violetteshev]

Looks like CUDA installation problem. Make sure you have CUDA 9.0 installed properly (guide) and check that you have correct paths in make.sh file. If it doesn't help, please refer to original faster-rcnn repo, I used their source code and installation scripts.

[Zhao-Yuting]

thank you so much.
i am sorry for that i use cuda10 now...so your code is incompatible with cuda10 right?

[violetteshev]

Yes, it works only with cuda 9.0

[gsrivas4]

@violetteshev I am planning to use your repo to get the bottom up features. My machine has CUDA 10.1 and pytorch version 1.5.0. Can you suggest what issues might occur, or you would have faced, if I try to upgrade your repo to the version on my machine. Would there be some api functionality, etc. or there are some libraries that your use which specifically need CUDA 9.0. Just wanted to take your advice before I start trying from my side, to desist from trying if it is not possible with your repo.

[violetteshev]

Hi, I used this faster rcnn implementation, and it only supports pytorch-0.4.0. They now have a branch that supports pytorch-1.0, so you can try to adjust the code. Or I'd suggest to use other implementations, like here the authors provided a docker image to run original caffe model.