Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: perception objects pointcloud better visualization #7095

Open
wants to merge 77 commits into
base: main
Choose a base branch
from

Conversation

KhalilSelyan
Copy link
Contributor

@KhalilSelyan KhalilSelyan commented May 22, 2024

Description

This PR adding new function to autoware_auto_perception_rviz_plugin. Now it has ability to publish point cloud of detected objects. It improve visualization of perception pipeline.
This feature can be enabled/disabled in control panel of the plugin by toggling Publish Objects Pointcloud property.
New point cloud plugin should be added to autoware config to visualize output of the plugin. Like it's shown in this video.

Output topics for point clouds:

/rviz2/output/predicted_objects_pointcloud
/rviz2/output/detected_objects_pointcloud
/rviz2/output/tracked_objects_pointcloud
Video demo with predicted objects from autoware demo bagfile.

Related links

This feature was discussed here #2860.

Tests performed

This feature was tested manually against sample rosbag from Rosbag replay simulation demo. I didn't find any reduces in performance.

Notes for reviewers

Pre-review checklist for the PR author

  • The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.

  • The PR follows the pull request guidelines.
    In-review checklist for the PR reviewers

  • The PR reviewers must check the checkboxes below before approval.

  • The PR follows the pull request guidelines.

  • The PR has been properly tested.

  • The PR has been reviewed by the code owners.
    Post-review checklist for the PR author
    The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

  • The PR is ready for merge.
    After all checkboxes are checked, anyone who has write access can merge the PR.

lexavtanke and others added 30 commits February 22, 2023 18:13
…d scripts

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…o_perception_plugin subscription to pointclouds and publishing pointclouds

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…message templated plugins, builds, but not woking correctly . Add rviz lauch file for debugging. modify package.xml

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…ntcloud_publisher' into feat/add_perception_objects_pointcloud_publisher
…o_rviz plugin to publish pointclouds

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…use objectsCallback to update objects

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…back to plugin

hardcoded topic names, one color for all classes of objects
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…ntcloud_publisher' into feat/add_perception_objects_pointcloud_publisher
…rent object classes

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…/disable publish poincloud

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…input pointcloud

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…and config

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
… pr error

Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…ntcloud_publisher' into feat/add_perception_objects_pointcloud_publisher
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
…ntcloud_publisher' into feat/add_perception_objects_pointcloud_publisher
Signed-off-by: Alexey Panferov <lexavtanke@gmail.com>
@KhalilSelyan KhalilSelyan linked an issue May 23, 2024 that may be closed by this pull request
6 tasks
@KhalilSelyan KhalilSelyan marked this pull request as ready for review May 23, 2024 09:54
@KhalilSelyan KhalilSelyan added component:perception Advanced sensor data processing and environment understanding. (auto-assigned) type:ui-ux User interface, user experience, graphical user interfaces. labels May 23, 2024
@KhalilSelyan KhalilSelyan changed the title Perception objects pointcloud better visualization feat: Perception objects pointcloud better visualization May 23, 2024
@github-actions github-actions bot removed the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label May 23, 2024
@kminoda
Copy link
Contributor

kminoda commented May 23, 2024

Very cool! Thank you for the PR. Let us assign one reviewer from TIER IV 🙏

@kminoda
Copy link
Contributor

kminoda commented May 23, 2024

@KhalilSelyan Would you first fix the CI, especially the pre-commit CI?

KhalilSelyan and others added 4 commits May 24, 2024 12:22
…on/predicted_objects_display.cpp

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>
…on/tracked_objects_display.cpp

Co-authored-by: kminoda <44218668+kminoda@users.noreply.github.com>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
@yukkysaito
Copy link
Contributor

yukkysaito commented May 28, 2024

Cool 👍
FYI
I am imagining here, but perhaps waymo is estimating the shape(bounding box) after doing the instance segmentation. Therefore, I don't think they are estimating pointcloud from shape.

@kminoda
Copy link
Contributor

kminoda commented May 28, 2024

@KhalilSelyan Sorry for keep you waiting. @ktro2828 will review your PR, thanks!

@ktro2828 ktro2828 changed the title feat: Perception objects pointcloud better visualization feat: perception objects pointcloud better visualization May 29, 2024
@ktro2828 ktro2828 self-requested a review May 29, 2024 05:47
KhalilSelyan and others added 2 commits May 30, 2024 00:52
Co-authored-by: Kotaro Uetake <60615504+ktro2828@users.noreply.github.com>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
…ottom line list

Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
@KhalilSelyan
Copy link
Contributor Author

@ktro2828 Have you had a chance to actually test the code in this PR ? It is still crashing rviz and haven't been able to debug the pointcloud implementation (i believe that's where the crashing comes from after some testing) Some help with that would be very appreciated

@ktro2828
Copy link
Contributor

ktro2828 commented Jun 3, 2024

@ktro2828 Have you had a chance to actually test the code in this PR ? It is still crashing rviz and haven't been able to debug the pointcloud implementation (i believe that's where the crashing comes from after some testing) Some help with that would be very appreciated

I'm testing now, but I've not encountered crashing yet.
Can you make this feature as an optional?

@technolojin
Copy link
Contributor

FYI
The current conflicts are due to the message changes from autoware_auto_msgs to autoware_msgs
#7239

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:common Common packages from the autoware-common repository. (auto-assigned) type:documentation Creating or refining documentation. (auto-assigned) type:ui-ux User interface, user experience, graphical user interfaces.
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Create better markers for tracked/predicted objects
7 participants