feat(perception_online_evaluator): add perception_online_evaluator

[kosuke55]

Description

Add perception_evaluator

This module allows for the evaluation of how accurately perception results are generated without the need for annotations. It is capable of confirming performance and can evaluate results from a few seconds prior, enabling online execution.

current metrics are

• Calculates lateral deviation between the predicted path and the actual traveled trajectory.

blue arrow are old predicted path and red arrows are actual history path the objects followed.

• Calculates lateral deviation between the smoothed traveled trajectory and the perceived position to evaluate the stability of lateral position recognition.
• Calculates yaw deviation between the smoothed traveled trajectory and the perceived position to evaluate the stability of yaw recognition.

smoothing path (window size=11)

rqt reconfigure

Tests performed

psim (max of time_horizonx=5)

perception_evaluator-2024-02-27_00.25.32.mp4

psim with rosbag(perception replyer) (max of time_horizonx=10)

perception_evaluator-2024-02-27_00.19.00.mp4

logging simulator

evaluator_description: feat/perception_evaluator
2024/02/27 https://evaluation.tier4.jp/evaluation/reports/1542d256-59b7-54c2-8d54-fc960508f9eb/?project_id=prd_jt

evaluator_description: feat/perception_evaluator time horizon 5
2024/02/27 https://evaluation.tier4.jp/evaluation/reports/ba008b2b-0242-52db-b18e-62781f6b554e/?project_id=prd_jt

evaluator_description: feat/perception_evaluator
2024/02/28 https://evaluation.tier4.jp/evaluation/reports/bbaa9fdc-cd30-56ef-b903-8e2908b1f674/?project_id=prd_jt

Effects on system behavior

Not applicable.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.
• The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

• The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

• There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

[kosuke55]

rename to perception_online_evaluator because it is confusing if considering https://github.com/tier4/driving_log_replayer/blob/develop/driving_log_replayer/scripts/perception_evaluator_node.py

checked it worked after rename

[rej55]

LGTM

[codecov]

Codecov Report

Attention: Patch coverage is 47.04492% with 448 lines in your changes are missing coverage. Please review.

Project coverage is 14.93%. Comparing base (4b784b4) to head (e203c3e).
Report is 7 commits behind head on main.

Files	Patch %	Lines
...evaluator/src/perception_online_evaluator_node.cpp	21.38%	61 Missing and 75 partials ⚠️
...tor/test/test_perception_online_evaluator_node.cpp	56.27%	7 Missing and 115 partials ⚠️
...eption_online_evaluator/src/metrics_calculator.cpp	50.94%	17 Missing and 87 partials ⚠️
...eption_online_evaluator/src/utils/marker_utils.cpp	32.05%	34 Missing and 19 partials ⚠️
...n_online_evaluator/src/utils/objects_filtering.cpp	80.76%	6 Missing and 4 partials ⚠️
...online_evaluator/src/metrics/deviation_metrics.cpp	46.66%	6 Missing and 2 partials ⚠️
...perception_online_evaluator/metrics_calculator.hpp	45.45%	0 Missing and 6 partials ⚠️
...uator/include/perception_online_evaluator/stat.hpp	45.45%	0 Missing and 6 partials ⚠️
...ude/perception_online_evaluator/metrics/metric.hpp	60.00%	0 Missing and 2 partials ⚠️
...ption_online_evaluator/utils/objects_filtering.hpp	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6493      +/-   ##
==========================================
+ Coverage   14.72%   14.93%   +0.21%     
==========================================
  Files        1900     1912      +12     
  Lines      130361   131180     +819     
  Branches    38381    38841     +460     
==========================================
+ Hits        19198    19596     +398     
- Misses      89647    89751     +104     
- Partials    21516    21833     +317     

Flag	Coverage Δ		*Carryforward flag
differential	47.04% <47.04%> (?)
total	14.72% <ø> (+<0.01%)	⬆️	Carriedforward from 4b784b4

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[xmfcx]

I've canceled the build-and-test-differential / build-and-test-differential (humble) job and restarted build-and-test-differential / build-and-test-differential (humble, -cuda) (pull_request) because this is the only required job from now.

This PR halves the time required for build-and-test-differential checks:

• ci(build-and-test-differential): remove non-cuda job #6495

If the test passes you can merge it. No need to do anything.

If you make any further changes to this PR, please rebase to main.

[kosuke55]

@xmfcx wow! Thank you very much!! I've been waiting for ci to finish for a long time...!

[kosuke55]

@takayuki5168 @TakaHoribe @tkimura4 (simulaor launch)
@miursh @YoshiRi @yukkysaito (perception launch)
could you please approve this PR as launch code owner?

[xmfcx]

@kosuke55 I'm sorry that this job is taking forever for so many people. I am trying to speed it up as much as I can.