Restructure bebop camera settings and ISP #2364
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kirkscheper
force-pushed
the
restructured_isp
branch
from
271e1d5
to
005a50b
Compare
gautierhattenberger
previously approved these changes
Dec 4, 2018
kirkscheper
force-pushed
the
restructured_isp
branch
from
005a50b
to
dfbd94b
Compare
kirkscheper
force-pushed
the
restructured_isp
branch
from
dfbd94b
to
8317e60
Compare
kirkscheper
force-pushed
the
restructured_isp
branch
from
8317e60
to
9acb599
Compare
dewagter
reviewed
Dec 7, 2018
dewagter
reviewed
Dec 7, 2018
There was a problem hiding this comment.
- great that many parameters are now computed
- great that the remaining parameters have documentation
- ok to have a bebop camera for that
- good to move the isp to bebop
- would be better to also compute default camera parameters
- should understand why this virtmem is needed
- should understand what the blanking is doing
|
Thanks, I will complete this in the new year after the stable release. |
kirkscheper
force-pushed
the
restructured_isp
branch
from
9acb599
to
3984255
Compare
OpenUAS
added
Enhancement
kirkscheper
force-pushed
the
restructured_isp
branch
from
3984255
to
b5f9c2e
Compare
kirkscheper
force-pushed
the
restructured_isp
branch
from
b5f9c2e
to
95c1071
Compare
2 tasks
tomvand
pushed a commit
that referenced
this pull request
Jun 17, 2019
This pull request updates the Bebop camera simulation in order to work with the new defines and settings from #2364. It also includes a number of small fixes by @kirkscheper . The new setup renders the entire camera sensor, which allows the crop and zoom settings to be changed at runtime just as on the real drone. The resolution of the simulated sensor can be reduced for better runtime performance. ---- * update gazebo sim camera * WIP Revert Bebop SDF cam hfov to 3.0, TODO update to full sensor size * WIP mt9f002.c remove OUTPUT_SCALER comment * WIP FIX image.c casting problem vectors.pos is now cast to float to prevent unwanted implicit casting to uint. I did not change the point_t types as I do not have time to test this, but this would be a good idea for a later pull request. * WIP lucas_kanade.c comment keep_bad_points * WIP opticflow_module.c replace memcpy with assignment for readability * WIP nps_fdm_gazebo fix mt9f117 check for bottom_cam instead of front_cam * WIP nps_fdm_gazebo do not set env_texture_size The current size (2048) is probably large enough for all realistic ZOOM settings. * WIP ERROR nps_fdm_gazebo fix camera, corrupt image in gazebo :/ * WIP ERROR zoom is not tuneable at runtime :/ * WIP fixed sensor resolution in gazebo Since the zoom should be adjustable at runtime, the zoom factor will need to be implemented in the image sampling routine, it cannot be set beforehand in the bebop model. * WIP include mt9f002.c and mt9v117.c in NPS Added simple wrapper files to prevent overwriting the video_thread_nps.c ones. * WIP enable runtime zoom and offset changes for mt9f002 in NPS! Zoom and offset settings can now be adjusted at runtime just like on the real drone. The mt9f002.c code handles the bounds checking etc. * WIP take out string comparison in read_image * WIP ignore GCC errors that do not appear in compilation for ARM Might be better to actually fix these errors, but I am not familiar with that part of the code. * WIP remove todo file * FIX ardrone2_gazebo build error linking error was caused by undefined reference to mt9f002 struct (which is not used in the ardrone). Removing the NPS MT9F002 define means that this struct is always linked, even though it may not be used. To fix, I added an empty `__attribute__((weak))` struct that prevents these linking errors, but could cause unexpected behavior when a Bebop model or other model with mt9f002 camera is used without the bebop_cam module. * FIX naming conflict after merge * FIX Allow arbitrary simulated sensor sizes for runtime improvement The FDM can now handle arbitrary sizes for the camera sensor. This allows the sensor size to be reduced from the real-world 4608x3288 to a lower resolution to reduce rendering times, at the cost of lower-resolution images. The downscaling of the sensor is controlled using the `NPS_MT9F002_SENSOR_RES_DIVIDER` define. * Update Bebop 2 gazebo models for new front- and bottom cam code
This was referenced Jul 8, 2019
noether
pushed a commit
to noether/paparazzi
that referenced
this pull request
Aug 10, 2019
This pull request updates the Bebop camera simulation in order to work with the new defines and settings from paparazzi#2364. It also includes a number of small fixes by @kirkscheper . The new setup renders the entire camera sensor, which allows the crop and zoom settings to be changed at runtime just as on the real drone. The resolution of the simulated sensor can be reduced for better runtime performance. ---- * update gazebo sim camera * WIP Revert Bebop SDF cam hfov to 3.0, TODO update to full sensor size * WIP mt9f002.c remove OUTPUT_SCALER comment * WIP FIX image.c casting problem vectors.pos is now cast to float to prevent unwanted implicit casting to uint. I did not change the point_t types as I do not have time to test this, but this would be a good idea for a later pull request. * WIP lucas_kanade.c comment keep_bad_points * WIP opticflow_module.c replace memcpy with assignment for readability * WIP nps_fdm_gazebo fix mt9f117 check for bottom_cam instead of front_cam * WIP nps_fdm_gazebo do not set env_texture_size The current size (2048) is probably large enough for all realistic ZOOM settings. * WIP ERROR nps_fdm_gazebo fix camera, corrupt image in gazebo :/ * WIP ERROR zoom is not tuneable at runtime :/ * WIP fixed sensor resolution in gazebo Since the zoom should be adjustable at runtime, the zoom factor will need to be implemented in the image sampling routine, it cannot be set beforehand in the bebop model. * WIP include mt9f002.c and mt9v117.c in NPS Added simple wrapper files to prevent overwriting the video_thread_nps.c ones. * WIP enable runtime zoom and offset changes for mt9f002 in NPS! Zoom and offset settings can now be adjusted at runtime just like on the real drone. The mt9f002.c code handles the bounds checking etc. * WIP take out string comparison in read_image * WIP ignore GCC errors that do not appear in compilation for ARM Might be better to actually fix these errors, but I am not familiar with that part of the code. * WIP remove todo file * FIX ardrone2_gazebo build error linking error was caused by undefined reference to mt9f002 struct (which is not used in the ardrone). Removing the NPS MT9F002 define means that this struct is always linked, even though it may not be used. To fix, I added an empty `__attribute__((weak))` struct that prevents these linking errors, but could cause unexpected behavior when a Bebop model or other model with mt9f002 camera is used without the bebop_cam module. * FIX naming conflict after merge * FIX Allow arbitrary simulated sensor sizes for runtime improvement The FDM can now handle arbitrary sizes for the camera sensor. This allows the sensor size to be reduced from the real-world 4608x3288 to a lower resolution to reduce rendering times, at the cost of lower-resolution images. The downscaling of the sensor is controlled using the `NPS_MT9F002_SENSOR_RES_DIVIDER` define. * Update Bebop 2 gazebo models for new front- and bottom cam code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is my effort at completing #2180. I implemented the changes that were made there (that I believed to be correct) as well as the recommended preset resolutions (VGA, 720p, 1080p and full resolution).
One of the changes was in how the camera settings were used, I built upon these suggestions and updated the settings. Now the user only sets the output resolution, image zoom and centre offsets and the rest is computed to generate a good output.
As the isp can only be used with the bebop, I moved the isp files to the bebop board folder.
In the process of doing this I realised that the documentation for the bebop cameras was quite poor and decided to make a module for the bebop cameras so that the settings can be auto documented. This results in an additional module to include in the airframe. To make this a little easier I changed all video_thread includes in bebop airframes to bebop_cam (which autoloads video_thread, plz let me know if you disagree or have an alternative).
There are also some miscellaneous changes that I found while implementing the upgrades (e.g. a setting for camera with the bebop_ae_awb even though it explicitly only works with the front camera).
Apologies for another large PR but I think that this simplifies and clarifies the camera usage for the bebop.