Change traffic signal interface to support V2X/V2I

[isamu-takagi]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

Hi, I'm thinking about V2X support in traffic signal interface. Currently, the planning component (traffic_light module in behavior_velocity_planner) has two traffic signal inputs, perception and external. It looks like we can input V2X information into external, but it was actually created for the purpose of overriding vehicle behavior by the operator. So this interface means the operator's decision such as pass and stop rather than the traffic signal color such as green or red, and the pass throuh logic described here is not performed.

Purpose

For the planning component, it doesn't matter if the input is image recognition results or V2X. Therefore, I'll unify the input so that V2X can be handled in the same way as image recognition results.

Possible approaches

The new architecture is as follows. Merge the image recognition results and V2X before inputting into the planning component. Typically the merge node will use V2X if it is available, otherwise it will use the most confident image recognition result. Also, as described later, light bulbs and traffic signals are treated separately.

The messages LightBulbsArray (or NewTrafficLightArray) and NewTrafficSignalArray (temporary name) are almost the same as TrafficSignalArray, except for the use of light bulbs ID and regulatory element ID. Or NewTrafficSignalArray could be changed to more semantic information such as {straight: green, turn_right: red}.

Map primitive IDs

There are multiple signals (as equipment) at one intersection, and image recognition targets them. On the other hand, V2X and the planning component only need one traffic signal (as information) in each direction of the intersection. This is defined in lanelet as light_blubs way ID and regulatory_element relation ID. See here for details of lanelet format.

Definition of done

• Add new interface message.
• Change the perception output interface.
• Change the planning input interface.
• Create the V2X output interface.
• Create the merge node from perception to planning.

[TakaHoribe]

For the planning component, it doesn't matter if the input is image recognition results or V2X. Therefore, I'll unify the input so that V2X can be handled in the same way as image recognition results.

I totally agree. The approach looks good to me as well.

[isamu-takagi]

@TakaHoribe Thank you. I would like to proceed with this proposal.

As a first step, create the autoware_perception_msgs package in autoware_msgs and define a message type as follows. Note that I changed the definition of traffic signal and traffic light. If there are other good words to distinguish between these, please let me know.

Type	Current implementation	Related lanelet ID
TrafficSignal.msg	none	Relation ID (type: regulatory_element)
TrafficLight.msg	TrafficSignal.idl	Way ID (type: traffic_light)
TrafficSignalElement.msg	TrafficLight.idl	Node ID if possible

Updated for #2567 (comment)

[isamu-takagi]

I've created a PR to add messages.
autowarefoundation/autoware_msgs#48

[yukkysaito]

cc @Mingyu1991 @miursh

[isamu-takagi]

The migration steps are as follows.

[yukkysaito]

@isamu-takagi The diagram above is a bit old, so the actual roles and such will be different.
@Mingyu1991 @miursh Could you share the updated figure?

[Mingyu1991]

@yukkysaito
Do you mean this figure?

@isamu-takagi
There would be two merge nodes within perception.
The "traffic_light_merger" would merge the results of the same traffic light from different cameras.
The "traffic_signal_arbitrator" would merge the results of traffic lights from the same group.

[isamu-takagi]

Thank you. Updated the diagram.

[isamu-takagi]

Step 1 is done. #2903

[stale]

This pull request has been automatically marked as stale because it has not had recent activity.

[KhalilSelyan]

@isamu-takagi Any updates ?

[isamu-takagi]

@KhalilSelyan
I asked the maintainers of each package to implement it. I hear they are working on it now.

• @TomohitoAndo for planning (step 2)
• @TomohitoAndo @Mingyu1991 for perception (step 3, 4)

[isamu-takagi]

Update the step 4 diagram to split merge step by targets. An implementation can perform multiple merge steps on the same node.

[isamu-takagi]

I noticed that TrafficLight.msg is converted to TrafficSignal.msg in perception component, and only TrafficSignal.msg is used for interface between components. So I will remove TrafficLight.msg.

[miursh]

Merged perception update #4050 #4125 #4176

This is the latest perception node diagaram.

[miursh]

@TakaHoribe @TomohitoAndo @isamu-takagi
Let me just check one thing. Are all works in this issue already done?
Can I close this issue?

[isamu-takagi]

@miursh I think so.
@TakaHoribe @TomohitoAndo @Mingyu1991 Are there any remaining tasks?

[TomohitoAndo]

@isamu-takagi @miursh
The changes for the planning modules haven't been merged yet. (PR)
The review for this PR is already done, but scenario simulator doesn't support the new interface yet, so we are waiting for it.
It's scheduled to be done by next week.

[TomohitoAndo]

Support for V2X in the scenario simulator has been delayed until early September. Thus, the PR for the planning modules is on hold until then.

[miursh]

Changing the node order of crosswalk_estimator
#4734

[TomohitoAndo]

@isamu-takagi @miursh @TakaHoribe
The changes for the planning modules have been merged, so I close this issue.
#4133