test surround view falied

[Vankeee]

Here is my command:

test-surround-view --module soft --input0 1.yuv --input1 2.yuv --input2 3.yuv --input3 4.yuv --output output.mp4 --in-w 640 --in-h 540 --out-w 1920 --out-h 640 --topview-w 1280 --topview-h 720 --scale-mode dualconst --frame-mode multi --save true --save-topview true --loop 1

I ran it on ubuntu 16.04， opencv 3.3.1

It turned out that ：

input0 file: 1.yuv
input1 file: 2.yuv
input2 file: 3.yuv
input3 file: 4.yuv
output file: output.mp4
input width: 640
input height: 540
output width: 1920
output height: 640
topview width: 1280
topview height: 720
scaling mode: dualconst
frame mode: multiframe
save output: true
save topview: true
loop count: 1
OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???'
XCAM INFO test-surround-view.cpp:620: calibration config path:./calib_params/
XCAM WARNING calibration_parser.cpp:201: open intrinsic file(./calib_params//intrinsic_camera_front.txt) failed.
XCAM ERROR test-surround-view.cpp:152: parse intrinsic params (./calib_params//intrinsic_camera_front.txt)failed.
XCAM ERROR test-surround-view.cpp:625: parse fisheye dewarp info(idx:0) failed.

What‘s wrong？ Sorry, I am a newbee.

[brmarkus]

Have you installed (either build by source or installing the libraries) the X264 encoder?
See e.g. here: "https://trac.ffmpeg.org/wiki/Encode/H.264" and "https://trac.ffmpeg.org/wiki/CompilationGuide".
Your example want to stitch multiple raw input streams together and encode/compress it into a MP4 container with an embedded h.264 compressed video stream - and misses the corresponding X264 encoding libraries.

[zongwave]

@Vankeee
you need to put your cameras' (front, right, rare and left) intrinsic & extrinsic calibration files under the test folder, use ocamcalib toolbox to do camera calibration:
https://sites.google.com/site/scarabotix/ocamcalib-toolbox

[pennluo]

@Vankeee
Currently, the camera calibration data only supports the file(format as txt) exported by Matlab's toolbox ocamcalib-toolbox, so if you use it directly, you need to use this toolbox to calibrate.

[pennluo]

@zongwave
I got Calibration files calib_results_front.txt ,as below:
#polynomial coefficients for the DIRECT mapping function (ocam_model.ss in MATLAB). These are used by cam2world
5 -2.612034e+02 0.000000e+00 1.111171e-03 -1.681827e-06 3.971074e-09
#polynomial coefficients for the inverse mapping function (ocam_model.invpol in MATLAB). These are used by world2cam
11 467.821131 299.560088 -35.564686 32.560224 34.181503 -53.088032 93.007996 315.194851 281.555355 109.639155 16.277912
#center: "row" and "column", starting from 0 (C convention)
238.809993 480.777383
#affine parameters "c", "d", "e"
1.000433 0.000121 0.000871
#image size: "height" and "width"
540 960

then I renamed calib_results_front.txt to intrinsic_camera_front.txt
What is the data format in extrinsic_camera_front.txt，Can you give some sample files?
Thanks!

[Vankeee]

@brmarkus
Here is my ffmpeg version information:
ffmpeg -version
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

You see the bold part, did I install the X264 encoder?

[zongwave]

@pennluo extrinsic_camera_front.txt record the camera's position and rotation information as below:
ocamcalib-toolbox calibration contains extrinsic results in "RRfin" field.

#extrinsic fisheye camera calibration, positions in millimeters, rotations in degrees
#Position x
xxxx
#Position y
yyyy
#Position z
zzzz
#Rotation x (roll)
r
#Rotation x (pitch)
p
#Rotation x (yaw)
y

[brmarkus]

@brmarkus
Here is my ffmpeg version information:
ffmpeg -version
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

You see the bold part, did I install the X264 encoder?

Maybe this helps: https://stackoverflow.com/questions/34024041/writing-x264-from-opencv-3-with-ffmpeg-on-linux
(looks like the codec-id's do not match; looks like your x264 installation is fine)

[pennluo]

@pennluo extrinsic_camera_front.txt record the camera's position and rotation information as below:
ocamcalib-toolbox calibration contains extrinsic results in "RRfin" field.

#extrinsic fisheye camera calibration, positions in millimeters, rotations in degrees
#Position x
xxxx
#Position y
yyyy
#Position z
zzzz
#Rotation x (roll)
r
#Rotation x (pitch)
p
#Rotation x (yaw)
y

Thanks!

[Vankeee]

@brmarkus
I used "!_writer.open (_file_name, 0x21, 30, frame_size))" to replace "!_writer.open (_file_name, cv::VideoWriter::fourcc ('X', '2', '6', '4'), 30, frame_size)", and the ffmpeg problem solved.

I still got "XCAM ERROR test-surround-view.cpp:625: parse fisheye dewarp info(idx:0) failed."
Do you know what is fisheye dewarp info? It's in the intrinsic_camera_front.txt file?

[pennluo]

@Vankeee
A few days ago,I met the same question at compiling,not modify any code,I compiled successfully
My solving method:
1.Make sure content format of calibration files(intrinsic&extrinsic) are correct
2.Checking path of calibration files,you can put it under "tests" folder
may be you can try

[pennluo]

@brmarkus @zongwave Hi
"RRfin" Field gives the pose of the camera relative to the checkerboard in each of the calibration images.
I want to use "RRfin" Field to get R(yaw)R(pitch)R(roll)T(x)T(y)T(z) of extrinsic_camera_XXX.txt
here is my thoughts:
Before starting calibration. I got 10 calibration images(img 0~9）
(img0）------ checkerboard placed on ground (z=0)
(img1-img9)--------checkerboad placed any 3D plane
After camera calibrate. we can get RRfin matrix 3x3x10 double
then I convert 3x3 RRfin[ ; ;1] to R(yaw)R(pitch)R(roll)T(x)T(y)T(z)
I wonder if I am right and how can I convert extrinsic matrix to Euler degree？
thanks！

[Vankeee]

@pennluo
Now I met another problem.I am using opencv.3.3.1, ubuntu 16.04. Do you use ubuntu? If you use ubuntu, then you should compile 'autofindconer'. Here is the description:
For Linux, please download the Windows version of the toolbox. To make use of the automatic checkerboard extraction, download and compile the following code: autoCornerFinder_linux.zip

When I compiled，it turned out that :
cvcalibinit3.cpp:34:37: fatal error: opencv2/core/internal.hpp: No such file or directory
#include <opencv2/core/internal.hpp>

I searched some, it maybe something about opencv version.

[zongwave]

@pennluo

How do I convert a rotation matrix to Euler angles?
http://web.archive.org/web/20041029003853/http://www.j3d.org/matrix_faq/matrfaq_latest.html#Q37

[Vankeee]

@pennluo
I gave up using matlab in ubuntu, I changed to Windows. Thanks!

[pennluo]

@zongwave @brmarkus Thanks a lot！

[pennluo]

@Vankeee
Calibration ： Windows + Ocamcalib-toolbox
Compiling ： Ubuntu 16.04 +OpenCV3.4

[Vankeee]

@pennluo
Could you leave your wechat? or QQ? Wanna chat with you about it= =, is it ok?
We can delete the comment later after I add u.

[wangqiang624731186]

@Vankeee can you share your extrinsic_camera_.txt and calib_results_.txt ?

[Saaloni]

HI @Vankeee . an you share your extrinsic_camera_.txt, calib_results_.txt and RRfin?