New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to change front camera resolution? #60

Open
zweistein opened this Issue Apr 8, 2013 · 27 comments

Comments

Projects
None yet
@zweistein

zweistein commented Apr 8, 2013

I'm using the Ar.Drone 2.0. with Firmware: 2.2.6.
The front camera can be used with resolution : 360p (640x360) or 720p (1280*720). The default setup is the first one, 360p. I would like to use the 720p.
How can I change it? Is there a parameter in the launch file?

Is there any update in camera togle issue?

@mani-monaj

This comment has been minimized.

Member

mani-monaj commented Apr 9, 2013

The front camera resolution can not be changed on the fly, because the image memory buffer size is now fixed. The dynamic memory allocation should fix the problem though. Here is how 720p video stream can be enabled on the current code. The glitch is, the driver should be recompiled.

  1. In file src/video.h:

Change MAX_STREAM_WIDTH and MAX_STREAM_HEIGHT to 1280 and 720 respectively.
Also, change D2_STREAM_WIDTH and D2_STREAM_HEIGHT (lines 32,33) to 1280 and 720 respectively.

  1. In file src/ardrone_sdk.cpp

Change ardrone_application_default_config.video_codec = H264_360P_CODEC (line 46) to H264_720P_CODEC

  1. Recompile with rosmake --pre-clean.

Please note that 1) The change in default video_codec may be overwritten by rosparam parameter of the same name. 2) This is a hack!

@truongngoctuan

This comment has been minimized.

truongngoctuan commented Jan 7, 2014

I followed your instruction, but there are many errors on decoding video stream. I don't know how to fix it

Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
I tried with increasing the bitrate... but useless

@mani-monaj

This comment has been minimized.

Member

mani-monaj commented Jan 8, 2014

Can you please paste the git diff of your changes?

@krisdestruction

This comment has been minimized.

krisdestruction commented Mar 7, 2014

The comment in /src/ardrone_drive.cpp says that there's no 720p support for now :/ is this true?

@mani-monaj

This comment has been minimized.

Member

mani-monaj commented Mar 16, 2014

@krisdestruction As I described in this thread, you can try 720p by changing the source code.

@mani-monaj mani-monaj added the question label Mar 16, 2014

@krisdestruction

This comment has been minimized.

krisdestruction commented Mar 16, 2014

I tried this and it seems to be spamming as noted above:
Decoding failed for a P Frame

@boris-il-forte

This comment has been minimized.

boris-il-forte commented Jul 25, 2014

same problem for me...

@thomas-bamford

This comment has been minimized.

thomas-bamford commented Apr 7, 2016

To increase video resolution to 720p, I tried the following steps as instructed above:

  1. In file include/ardrone_autonomy/video.h:

Change MAX_STREAM_WIDTH and MAX_STREAM_HEIGHT (lines 33, 34) to 1280 and 720 respectively.
Also, change D2_STREAM_WIDTH and D2_STREAM_HEIGHT (lines 58, 59) to 1280 and 720 respectively.

  1. In file src/ardrone_sdk.cpp:

Change ardrone_application_default_config.video_codec = H264_360P_CODEC (line 71) to H264_720P_CODEC

  1. Recompile with catkin_make.

The results after compilation and launching ardrone.launch with an AR.Drone 2.0 connected on wifi are shown below with an image saved from the /ardrone/image_raw rostopic.

...
Starting thread video_stage
Starting thread video_recorder

video stage thread initialisation

Video multisocket : init 2 sockets
Video multisocket : connecting socket 0 on port 0 UDP
Video recorder thread initialisation
Video multisocket : connecting socket 1 on port 5555 TCP
...
PA : MEMORY SPACE ALLOWED : 40 MB
...
Sending default CAT_APPLI settings
Sending default CAT_USER settings
Sending default CAT_SESSION settings
Decoding failed for a I Frame
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
...
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
[ INFO] [1460046443.794771981]:   SEND: CAT_COMMON/outdoor = 1.000000 (DEFAULT = 0.000000)
[ INFO] [1460046443.794889905]:   SEND: CAT_COMMON/enemy_colors = 3.000000 (DEFAULT = 1.000000)
[ INFO] [1460046443.794940702]:   SEND: CAT_APPLI/navdata_options = 268435456.000000 (DEFAULT = 65537.000000)
[ INFO] [1460046443.795005714]:   SEND: CAT_USER/euler_angle_max = 0.210000 (DEFAULT = 0.209440)
[ INFO] [1460046443.795087181]:   SEND: CAT_USER/control_yaw = 1.750000 (DEFAULT = 1.745329)
[ INFO] [1460046443.795192413]:   SEND: CAT_SESSION/video_codec = 131.000000 (DEFAULT = 32.000000)
[ INFO] [1460046443.795296483]:   SEND: CAT_APPLI/bitrate = 4000.000000 (DEFAULT = 1000.000000)
[ INFO] [1460046443.795445789]:   SEND: CAT_SESSION/max_bitrate = 4000.000000 (DEFAULT = 1000.000000)
[ INFO] [1460046443.795539563]:   SEND: CAT_SESSION/detect_type = 10.000000 (DEFAULT = 3.000000)
[ INFO] [1460046443.795625731]:   SEND: CAT_SESSION/detections_select_h = 32.000000 (DEFAULT = 0.000000)
[ INFO] [1460046443.795717897]:   SEND: CAT_SESSION/detections_select_v_hsync = 128.000000 (DEFAULT = 0.000000)
Decoding failed for a P Frame
Decoding failed for a P Frame
[ INFO] [1460046443.835659236]: Successfully connected to 'My ARDrone' (AR-Drone 2.0 - Firmware: 2.4.8) - Battery(%): 86
[ INFO] [1460046443.835779725]: Navdata Publish Settings:
[ INFO] [1460046443.835858745]:     Legacy Navdata Mode: On
[ INFO] [1460046443.835951245]:     ROS Loop Rate: 50 Hz
[ INFO] [1460046443.836020931]:     Realtime Navdata Publish: Off
[ INFO] [1460046443.836095017]:     Realtime Video Publish: Off
[ INFO] [1460046443.836165093]:     Drone Navdata Send Speed: 200Hz (navdata_demo=0)
Decoding failed for a P Frame
Decoding failed for a P Frame
...
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
...

frame0001

@mani-monaj

This comment has been minimized.

Member

mani-monaj commented Dec 7, 2016

@thomas-bamford Oh, I missed this update. Did you manage to fix the issue?

@Fawzikhattar

This comment has been minimized.

Fawzikhattar commented Mar 3, 2017

Has anyone managed to fix this issue?

@thomas-bamford

This comment has been minimized.

thomas-bamford commented Mar 3, 2017

Sorry, was not able to fix this issue. I upgraded my hardware to the Parrot Bebop 2 and it streams at 720p.

@Fawzikhattar

This comment has been minimized.

Fawzikhattar commented Mar 11, 2017

I was able to find a hack to it (although sometimes the H264 decoder needs some time to start decoding properly).

In addition to the steps above add the following in the video_stage_tcp.c file in devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video

function declaration:

static void changeSPS(uint8_t * buf)
{ int i=0;
  int dontbreak=1;
  int numberOfBytesToMove=20;
   while(dontbreak)
      {
        if(buf[i]==0x67)
        { dontbreak=0;
         memmove (buf+i+1, buf+i,numberOfBytesToMove);
     
        }
   i=i+1;
       }
}

inside function video_stage_tcp_transform add this after the first two lines:

if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]);

That did it for me although sometimes the decoder takes some time to work properly.

@mani-monaj

This comment has been minimized.

Member

mani-monaj commented Mar 13, 2017

@Fawzikhattar Thank you for the update. Does this change break regular (non-720p) streaming in your tests?

@Fawzikhattar

This comment has been minimized.

Fawzikhattar commented Mar 13, 2017

I didn't test if it breaks it or n as im only intrested in the 720p video stream. However a simple if condition on the codec used can avoid this problem in case it exist. Also note that a better more robust way to solve this problem might exist (I dont have too much experience in decoding videos).

@shlapion

This comment has been minimized.

shlapion commented May 14, 2017

Hello,
I have tried @Fawzikhattar method and it works. As mentioned the decoder needs some time to start properly decoding. In my test took around (45s-1m). After that the video is stable.
I also recommend a re-calibration of the camera to have it work properly.

Personally, I have little to no experience in decoding videos, so I also can't recommend a better approach to solving this problem.

@mani-monaj do you think this issue could be addressed in the near future?

@Fawzikhattar many thanks for your solution !

@chinnapong

This comment has been minimized.

chinnapong commented May 15, 2017

The problem comes from the nature of H.264 codec at 720P resolution. The decoder cannot start unless the SPS and PPS frames are properly arrived.

The ARDone sends SPS and PPS frame regularly so, normally, there won't be a problem. However, the actual problem lies in the library that AR_Drone autonomy used for decoding the video, which is the FFMPEG.

We can say that it's the bug of FFMPEG library (or there may be some reason why the programmer of FFMPEG leaves this bug inside). AR-Drone (and most of the H.264 streaming application) sends the SPS and PPS in the first 20 bytes within every IDR frame. The location of these SPS and PPS are in the first 20 bytes of the frame. However, FFMPEG expects that the first byte of the IDR frame should always be 0, and the SPS and PPS should starts from the second byte.

You can see that the function "ChangeSPS" as provided by Fawzi works by shifting the first 20 bytes of the video frame to the right by one byte. Thus, FFMPEG can finally receive the SPS, PPS properly for decoding the video.

The resason why sometimes it takes 45-1 minutes before the video could be start is because it's the period where the drone does not send the IDR frame yet. We cannot do any much about it because the frequency of sending IDR frame is hard-coded and cannot be controlled. ><"

I hope that my explanation helps.

@shlapion

This comment has been minimized.

shlapion commented May 17, 2017

Hello @chinnapong ,

thank you for the detailed explanation.
Helped me to understand the issue better: ).

@Kikkomann

This comment has been minimized.

Kikkomann commented Jun 7, 2017

Is it only me that can't make it work? I get the same result as @thomas-bamford had.
I have followed @mani-monaj and @Fawzikhattar guides. Are there other some implicit details that I should know (I am new to ROS and c/c++)?

@Fawzikhattar

This comment has been minimized.

Fawzikhattar commented Jun 7, 2017

Did you recompile and clean the project after all the changes? if yes how?

@Kikkomann

This comment has been minimized.

Kikkomann commented Jun 8, 2017

I have tried rosmake --pre-clean in the ardron_autonomy folder and make in the drone root folder (containing ardrone_autonomy and my own drone project).

@cyberphantom

This comment has been minimized.

cyberphantom commented Feb 5, 2018

I've tried the steps above. I got 4 x 640x360 frames inside a 1280x720 (as the same in the comment of @thomas-bamford above). Please update the correct steps to get this resolution change works, or advise if this is not a practical solution.

Thanks,

@cyberphantom

This comment has been minimized.

cyberphantom commented Feb 5, 2018

It worked after rosmake --pre-clean

@TengFeiHan0

This comment has been minimized.

TengFeiHan0 commented Jun 4, 2018

@cyberphantom
if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]);
I can't understand what did he mean "inside function video_stage_tcp_transform add this after the first two lines:"
could you tell me the specific location where these codes should add?

@cyberphantom

This comment has been minimized.

cyberphantom commented Jun 11, 2018

Go to "video_stage_tcp.c" file inside "devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video" directory (inside your catkin for instance)

Go to "C_RESULT video_stage_tcp_transform(video_stage_tcp_config_t *cfg, vp_api_io_data_t *in, vp_api_io_data_t *out)" function

it has only two lines and a comment.

Add "if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]);" just inside it before the comment

@bharat-robotics

This comment has been minimized.

bharat-robotics commented Jun 29, 2018

Hello @cyberphantom @Fawzikhattar,

I am getting buffer full issue and also the ardrone_driver node crashes after like 2-3 minutes. Did you get the same problem? If yes, how did you solve the problem?

@cyberphantom

This comment has been minimized.

cyberphantom commented Jun 30, 2018

No actually, I remember running it for a long time without any issues.

@bharat-robotics

This comment has been minimized.

bharat-robotics commented Jul 5, 2018

@mani-monaj I am getting buffer full issue and it's coming from video_stage_tcp.c file in devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video, probably due to high resolution images. I tried to increase the buffer size at line 62 ( BUFFER_SIZE=cfg->maxPFramesPerIFrame * cfg->frameMeanSize ) by multiplying by 4. But it's giving me segmentation fault. I am not sure what exactly the buffer size should be. Can you look into this, please?
screenshot from 2018-07-05 16-40-47

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment