Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I use the model with my own image stream and pose data? #67

Open
moaazassali opened this issue Mar 24, 2023 · 0 comments
Open

How do I use the model with my own image stream and pose data? #67

moaazassali opened this issue Mar 24, 2023 · 0 comments

Comments

@moaazassali
Copy link

moaazassali commented Mar 24, 2023

I think I might be over complicating the process, but here is what I tried so far:

I have tried calling the Neural_SLAM_Module() alone first to test out using my own images, but I can't seem to get that to work let alone the other models for exploration.

test.py file:

from model import RL_Policy, Local_IL_Policy, Neural_SLAM_Module
from PIL import Image
import torch
import numpy as np
from arguments import get_args

device = torch.device("cuda:0")

map_size_cm = 1000
map_resolution_cm = 5
map_size = map_size_cm // map_resolution_cm

full_w, full_h = map_size, map_size
local_w, local_h = full_w // 4, full_h // 4

# Initializing full and local map
full_map = torch.zeros(1, 4, full_w, full_h).float().to(device)
local_map = torch.zeros(1, 4, local_w, local_h).float().to(device)

# Initial full and local pose
full_pose = torch.zeros(1, 3).float().to(device)
local_pose = poses = torch.zeros(1, 3).float().to(device)

img = Image.open("test_img.jpg")

img = np.transpose(img, (2, 0, 1))

obs = torch.from_numpy(np.transpose(np.expand_dims(img, axis=0), (0, 1, 3, 2))).float().to(device)
last_obs = obs

print(obs.size()) # (1,3,128,128)

args = get_args()
args.device = device
args.frame_height = 128 # image height
args.frame_width = 128 # image height
args.map_resolution = map_resolution_cm
args.map_size_cm = map_size


nslam_module = Neural_SLAM_Module(args).to(device)
_, _, local_map[:, 0, :, :], local_map[:, 1, :, :], _, local_pose = \
                nslam_module(last_obs, obs, poses, local_map[:, 0, :, :],
                             local_map[:, 1, :, :], local_pose, build_maps=True)

This gives me the following error:

/home/.../Desktop/Neural-SLAM/test.py:27: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1678402411778/work/torch/csrc/utils/tensor_numpy.cpp:206.)
obs = torch.from_numpy(np.transpose(np.expand_dims(img, axis=0), (0, 1, 3, 2))).float().to(device)
torch.Size([1, 3, 128, 128])
Auto GPU config:
Number of processes: 24
Number of processes on GPU 0: 8
Number of processes per GPU: 16
/home/.../anaconda3/envs/neuralslam/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
/home/.../anaconda3/envs/neuralslam/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=ResNet18_Weights.IMAGENET1K_V1. You can also use weights=ResNet18_Weights.DEFAULT to get the most up-to-date weights.
warnings.warn(msg)
/home/.../anaconda3/envs/neuralslam/lib/python3.10/site-packages/torch/nn/functional.py:4298: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn(
Traceback (most recent call last):
File "/home/.../Desktop/Neural-SLAM/test.py", line 44, in
nslam_module(last_obs, obs, poses, local_map[:, 0, :, :],
File "/home/.../anaconda3/envs/neuralslam/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/.../Desktop/Neural-SLAM/model.py", line 209, in forward
rot_mat, trans_mat = get_grid(st_poses,
File "/home/.../Desktop/Neural-SLAM/utils/model.py", line 40, in get_grid
rot_grid = F.affine_grid(theta1, torch.Size(grid_size))
File "/home/.../anaconda3/envs/neuralslam/lib/python3.10/site-packages/torch/nn/functional.py", line 4341, in affine_grid
return torch.affine_grid_generator(theta, size, align_corners)
RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [1, 3] but got: [24, 3].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant