Getting some errors with the inference

[FlowDownTheRiver]

Hi there,thanks for the work.But getting some errors....

packages in environment at conda\envs\realbasicvsr:

Name Version Build Channel

absl-py 1.0.0 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
blas 1.0 mkl
ca-certificates 2021.10.26 haa95532_2
cachetools 4.2.4 pypi_0 pypi
certifi 2021.10.8 py37haa95532_0
charset-normalizer 2.0.10 pypi_0 pypi
click 7.1.2 pypi_0 pypi
colorama 0.4.4 pypi_0 pypi
cudatoolkit 10.1.243 h74a9793_0
freetype 2.10.4 hd328e21_0
google-auth 2.3.3 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
grpcio 1.43.0 pypi_0 pypi
idna 3.3 pypi_0 pypi
imageio 2.13.5 pypi_0 pypi
importlib-metadata 4.10.0 pypi_0 pypi
intel-openmp 2021.4.0 haa95532_3556
jpeg 9b hb83a4c4_2
libpng 1.6.37 h2a8f88b_0
libtiff 4.2.0 hd0e1b90_0
libuv 1.40.0 he774522_0
libwebp 1.2.0 h2bbff1b_0
lmdb 1.3.0 pypi_0 pypi
lz4-c 1.9.3 h2bbff1b_1
markdown 3.3.6 pypi_0 pypi
mkl 2021.4.0 haa95532_640
mkl-service 2.4.0 py37h2bbff1b_0
mkl_fft 1.3.1 py37h277e83a_0
mkl_random 1.2.2 py37hf11a4ad_0
mmcv-full 1.4.2 pypi_0 pypi
mmedit 0.12.0 pypi_0 pypi
model-index 0.1.11 pypi_0 pypi
networkx 2.6.3 pypi_0 pypi
ninja 1.10.2 py37h559b2a2_3
numpy 1.21.2 py37hfca59bb_0
numpy-base 1.21.2 py37h0829f74_0
oauthlib 3.1.1 pypi_0 pypi
olefile 0.46 py37_0
opencv-python-headless 4.5.4.60 pypi_0 pypi
openmim 0.1.5 pypi_0 pypi
openssl 1.1.1l h2bbff1b_0
ordered-set 4.0.2 pypi_0 pypi
packaging 21.3 pypi_0 pypi
pandas 1.3.5 pypi_0 pypi
pillow 8.4.0 py37hd45dc43_0
pip 21.2.4 py37haa95532_0
protobuf 3.19.1 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pyparsing 3.0.6 pypi_0 pypi
python 3.7.11 h6244533_0
python-dateutil 2.8.2 pypi_0 pypi
pytorch 1.7.1 py3.7_cuda101_cudnn7_0 pytorch
pytz 2021.3 pypi_0 pypi
pywavelets 1.2.0 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
regex 2021.11.10 pypi_0 pypi
requests 2.27.1 pypi_0 pypi
requests-oauthlib 1.3.0 pypi_0 pypi
rsa 4.8 pypi_0 pypi
scikit-image 0.19.1 pypi_0 pypi
scipy 1.7.3 pypi_0 pypi
setuptools 58.0.4 py37haa95532_0
six 1.16.0 pyhd3eb1b0_0
sqlite 3.37.0 h2bbff1b_0
tabulate 0.8.9 pypi_0 pypi
tensorboard 2.7.0 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tifffile 2021.11.2 pypi_0 pypi
tk 8.6.11 h2bbff1b_0
torchaudio 0.7.2 py37 pytorch
torchvision 0.8.2 py37_cu101 pytorch
typing_extensions 3.10.0.2 pyh06a4308_0
urllib3 1.26.7 pypi_0 pypi
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
werkzeug 2.0.2 pypi_0 pypi
wheel 0.37.0 pyhd3eb1b0_1
wincertstore 0.2 py37haa95532_2
xz 5.2.5 h62dcd97_0
yapf 0.32.0 pypi_0 pypi
zipp 3.7.0 pypi_0 pypi
zlib 1.2.11 h8cc25b3_4
zstd 1.4.9 h19a0ad4_0

---

For pictures I ran the test code :

(realbasicvsr) C:\RealBasicVSR>python inference_realbasicvsr.py configs/realbasicvsr_x4.py checkpoints/RealBasicVSR_x4.pth data/demo_000 results/demo_000
2022-01-07 06:36:34,070 - mmedit - INFO - load checkpoint from torchvision path: torchvision://vgg19
load checkpoint from local path: checkpoints/RealBasicVSR_x4.pth

it did nothing.

For video I ran the test code setting the --max_seq_len=2 :

(realbasicvsr) C:\RealBasicVSR>python inference_realbasicvsr.py configs/realbasicvsr_x4.py checkpoints/RealBasicVSR_x4.pth data/demo_001.mp4 results/demo_001.mp4 --max_seq_len=2 --fps=12.5
2022-01-07 06:38:02,236 - mmedit - INFO - load checkpoint from torchvision path: torchvision://vgg19
load checkpoint from local path: checkpoints/RealBasicVSR_x4.pth
Traceback (most recent call last):
File "inference_realbasicvsr.py", line 144, in
main()
File "inference_realbasicvsr.py", line 130, in main
cv2.destroyAllWindows()
cv2.error: OpenCV(4.5.4) D:\a\opencv-python\opencv-python\opencv\modules\highgui\src\window.cpp:1268: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvDestroyAllWindows'

it gave this error.

---

For video I ran the test code default :

(realbasicvsr) C:\RealBasicVSR>python inference_realbasicvsr.py configs/realbasicvsr_x4.py checkpoints/RealBasicVSR_x4.pth data/demo_001.mp4 results/demo_001.mp4 --fps=12.5
2022-01-07 06:40:14,850 - mmedit - INFO - load checkpoint from torchvision path: torchvision://vgg19
load checkpoint from local path: checkpoints/RealBasicVSR_x4.pth
Traceback (most recent call last):
File "inference_realbasicvsr.py", line 144, in
main()
File "inference_realbasicvsr.py", line 117, in main
outputs = model(inputs, test_mode=True)['output'].cpu()
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmcv\runner\fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\restorers\srgan.py", line 95, in forward
return self.forward_test(lq, gt, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\restorers\real_esrgan.py", line 211, in forward_test
output = _model(lq)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\real_basicvsr_net.py", line 87, in forward
outputs = self.basicvsr(lqs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\basicvsr_net.py", line 126, in forward
flows_forward, flows_backward = self.compute_flow(lrs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\basicvsr_net.py", line 98, in compute_flow
flows_backward = self.spynet(lrs_1, lrs_2).view(n, t - 1, 2, h, w)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\basicvsr_net.py", line 346, in forward
input=self.compute_flow(ref, supp),
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\basicvsr_net.py", line 315, in compute_flow
], 1))
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmedit\models\backbones\sr_backbones\basicvsr_net.py", line 420, in forward
return self.basic_module(tensor_input)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\container.py", line 117, in forward
input = module(input)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\mmcv\cnn\bricks\conv_module.py", line 201, in forward
x = self.conv(x)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\conv.py", line 423, in forward
return self._conv_forward(input, self.weight)
File "C:\Users\breakmycurse.conda\envs\realbasicvsr\lib\site-packages\torch\nn\modules\conv.py", line 420, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDA out of memory. Tried to allocate 4.35 GiB (GPU 0; 11.00 GiB total capacity; 4.37 GiB already allocated; 2.46 GiB free; 7.07 GiB reserved in total by PyTorch)

it gave an oom error.

---

System is Win 10 64 bit, 1080 ti =11 gb.
model is in the right folder and the environment is done with conda with the given commands in order.

[ckkelvinchan]

It is likely that the GPU is unable to accommodate the long sequences. What is the spatial size of the frames and how many frames are in the sequence?

[FlowDownTheRiver]

I was using the test images and the video contained with the repo to test.Haven't tried anything else

[ckkelvinchan]

You can try to separate the frames into short segments by specifying --max_seq_len. For example, you can try

python inference_realbasicvsr.py configs/realbasicvsr_x4.py checkpoints/RealBasicVSR_x4.pth data/demo_000 results/demo_000 --max_seq_len 20

[FlowDownTheRiver]

--max_seq_len=2 I have given the argument to the command line as this, cos in the repo page it was written that way. So this clarifies the problem. There is no '=' there. Will test it when I get home and see if it works or not. Thank you very much for your time and answer. Will get back to you after testing.

[FlowDownTheRiver]

Ok now both images and video inference are working.I am sorry for my mistake.It was at the user end. Thank you very much.