-
Notifications
You must be signed in to change notification settings - Fork 343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Depth Image from unity causing the unity bridge connection to crash! EDIT: Need camera Calibration Matrix for depth image to point cloud generation #47
Comments
I was facing a similar problem last week. Will really appreciate a solution for this one |
Been facing a similar issue with the rgb camera feed. Any feedback would be a massive help. |
We will look into it, and solve it as soon as possible. (within this week) |
@Stavya1993 https://github.com/uzh-rpg/flightmare/blob/dev/samples/flightros/src/camera/camera.cpp I've written an example of how to publish the images on a ROS topic. This sample is written for the soon published binary release and needs minor modification to work with the previous release. I did not encounter any unity_bridge connection crashes with this code. I hope the sample helps you! |
@slimeth: Thanks so much, I am really surprised, I followed a very similar code with and without using image transport. I perhaps have one last question around this topic. What is the camera calibration values in this case? I found the depth scale, but also need the intrinsic matrix and cant seem to find it. |
@slimeth thanks for resolving the previous issue. I also wanted the camera calibration matrix for the camera spawned in flightmare. wasn't able to find it ? Your help will be really appreciated |
Hi, Ive managed to resolve everything I need for my task except the calibration matrix. Any help in this will be highly appreciated. |
Hi, you don't extract the calibration matrix from Unity, instead, it is calculated on the client-side (ROS/C++). Since both the image dimensions (width x height) and field of view (FOV) are defined by the user. (see https://github.com/uzh-rpg/flightmare/blob/master/flightlib/include/flightlib/sensors/rgb_camera.hpp#L59-L61) you can compute the focal length using this formula: |
Perfect! Thank you so so much! Sorry for bothering so much again. Think what you guys have made is absolutely beautiful! |
Hi @Stavya1993 . May I know if you are able to retrieve images in RVIZ by subscribing those topics? |
Yes @BananaGorilla98 |
@Stavya1993 would you mind to share how you do it? I was trying the method mentioned here but still could not get anything in my RVIZ. |
Sure. |
FLIGHT_PILOT_NODE for obtaining depth camera image
|
Hi, The flight_pilot has unity ready and unity render as private instantiations in the flight_pilot.h file in include. Move them to public instantiations and it should work. |
@Stavya1993 Thanks for sharing your code! I have tried your code and it built successfully. However, the RVIZ still not able to show the images. May I know if there is any configuration that I need to do? |
As per the code given in test/unity_bridge :
I followed a similar logic to build a publisher from my flight_pilot_node. However the unity bridge connection seems to crash after only 4-5 published messages. I even used image transporter to compress my messages and tried giving a rate with sufficient sleep duration to ensure that my data overflow was not causing issues. However, I haven't achieved much success till now.
Sharing my code edits here:
flight_pilot_node.cpp:
flight_pilot.hpp
void depthCamera(ros::Publisher msg);
// Depth camera variables - cv::Mat depth_image; bool handle_output;
flight_pilot.cpp:
Function depthCamera added
This is an extra time - sensitive task and would really appreciate if I can get some help in understanding if I am doing something wrong or is this a unity-ros issue that I cannot account for.
Regards
The text was updated successfully, but these errors were encountered: