Running openpilot in Carla, manual control and model inference is working, but control is not

[dvorak0]

Describe the bug

I run openpilot with carla simulation, and end up with openpilot not able to control the car.

1. launch start_carla.sh
2. launch launch_openpilot.sh
3. launch bridge.py --dual_camera
4. press w for a few times. when the speed reachs 40km/s, press 1 for a few times. expect to see openpilot engaged and control the car.

Thanks for help.

OS Version

run everything inside docker(with ubuntu_setup.sh setup)

openpilot version or commit

3555a08

Additional info

I installed the libzmq from source, otherwise the onnx_runner will crash.

I also tried pressing 1 when the car is still, and get the same results(showing openpilot unavailable).

[mkjooriah]

Hello, I am also experiencing the same issue.

OS Version

Ubuntu server 20.04 LTS

openpilot version or commit

85908f7

Additional info

I also installed libzmq from source.

[hugopinho0]

I have the same problem.

Does anyone have more suggestions of what I can do to understand the problem or other things to test?

[youdaorobot]

I have the same problem, I can manual control the car, but press 1 ,always show "OpenPilot Unavailable press set to engage"

[dvorak0]

I finally got some some time to solve it. I write it down, hope it helpful to others.

After reading a lot code but still fail to figure out the state machine, the test in https://github.com/commaai/openpilot/blob/master/tools/sim/tests/test_carla_integration.py stands up.

The failure in this test help me find two issues preventing the openpilot to engage:

1. the control rate is lower than 100hz, then I modify

   openpilot/selfdrive/controls/controlsd.py

   Line 207 in 16f0f15

   self.rk = Ratekeeper(100, print_delay_threshold=None)

   to 50Hz.
2. I mistakely turn off "Disengage on Accelerator Pedal" in openpilot's mobile UI's 'Toggle' menu. Then I turned it on.

right now, I can run close-loop openpilot in Carla.

[SyazwanDev]

Hi, I've tried your solution and it seems not working for me. May I see your second terminal where you run your ./launch_openpilot.sh? Because it seems the modeld keeps crashing on my part. Did your modeld turns out red as well?

[dvorak0]

Hi, I've tried your solution and it seems not working for me. May I see your second terminal where you run your ./launch_openpilot.sh? Because it seems the modeld keeps crashing on my part. Did your modeld turns out red as well?

I fixed model inference by following #23666 (comment) to install libzmq. As a result, the CUDAExecutionProvider is used.

selfdrive/modeld/modeld.cc: models loaded, modeld starting
selfdrive/modeld/modeld.cc: connected main cam with buffer size: 3493536 (1928 x 1208)
selfdrive/modeld/modeld.cc: connected extra cam with buffer size: 3493536 (1928 x 1208)
selfdrive/modeld/modeld.cc: skipping model eval. Dropped 229 frames
['/openpilot/selfdrive/modeld/runners/onnx_runner.py', 'models/supercombo.onnx', '']
Onnx available providers:  ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Onnx selected provider:  ['CUDAExecutionProvider']
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
{"event": "fingerprinted", "car_fingerprint": "HONDA CIVIC 2016", "source": 2, "fuzzy": false, "cached": false, "fw_count": 0, "ecu_responses": [], "vin_rx_addr": 0, "error": true}
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
Onnx using  ['CUDAExecutionProvider', 'CPUExecutionProvider']
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
clocksd logcatd proclogd logmessaged modeld ui soundd locationd calibrationd torqued controlsd deleter laikad navd paramsd plannerd radard thermald statsd
ready to run onnx model ['input_imgs', 'big_input_imgs', 'desire', 'traffic_convention', 'features_buffer'] [[1, 12, 128, 256], [1, 12, 128, 256], [1, 100, 8], [1, 2], [1, 99, 128]]
selfdrive/modeld/modeld.cc: skipping model eval. Dropped 92 frames
{"event": "commIssue", "error": true, "invalid": ["liveTorqueParameters"], "not_alive": ["testJoystick", "driverCameraState"], "not_freq_ok": ["testJoystick", "driverCameraState"], "can_rcv_timeout": false}

[SyazwanDev]

Hi, thank you for the reply. Yes, I've tried that and my modeld is not crashing anymore (even though it do actually crash sometimes, depends on my luck but better than before). However, then my simulator shows "Vehicle Parameter Identification Failed" and I solve it using this #27282 Hope it helps anyone who is facing the same issue.

[tomdries]

I also installed libzmq from source, which fixed the modeld problem. I'm still stuck at the Openpilot Unavailable problem however. The suggestion by @dvorak0 did not work, and I keep getting Openpilot unavailable message as soon as I press "1". Interestingly, when I run it using the --joystick option and I press the square button on my steering wheel, the car starts accelerating, and it correctly steers itself halfway through the first corner. Then, I get the error "Vehicle Parameter Identification Failed". Also, the rest of the steering wheel controls does not work.

[SyazwanDev]

If the error is "Vehicle Parameter Identification Failed", try this solution #27282. Then, to engage the openpilot, make sure that "Disengage with Pedal" on the OpenPilot UI toggle option is enabled. You also need to press button '1' and '2' alternately for few times to engage the OpenPilot (not just button '1' alone).

[ev1anzzZ]

I also encounter a similar problem. When I use --joystick option, it is able to turn on and off the cruise mode of openpilot through the buttons of my steering wheel Logitech g29. But the cruise mode seems not very effective, the vehicle accelerate from the starting point, begin to turn right and hit a wall. In addition, the steering ,throttle and brake function of my steering wheel cannot be used when i exit the cruise mode, and I am sure these functions perform well in the jstest-gtk driver.

[adeebshihadeh]

Check out our new simulator, MetaDrive. It works a lot better.