Commander refuses to arm, no error message/reason

[mhkabir]

I digged deeped into the issue in #3122, and it seems that this is an independent problem.

After full system setup (calibration and everything), once I reboot, the LED flashes red on startup and refuses to let me arm (symptom of failing pre-flight checks or incomplete boot ?)

QGC shows this :

[22:43:28.549 - COMP:1] Critical: INVAL: ARMING_STATE_INIT - ARMING_STATE_ARMED
[22:43:28.600 - COMP:1] Critical: REJECTING component arm cmd
[22:43:28.651 - COMP:1] Critical: command temporarily rejected: 400

No reason for rejection is shown.

Entering via USB console and doing a commander stop-start results in the system going to standby mode with the normal blue LED heatbeat.

@LorenzMeier Any ideas? I'm running a Pixhack board, and it's rather difficult to access the serial console for now, since it is mounted deep inside the airframe. Is there any way to dump bootlogs/console output to a file on the SD card?

[RomanBapst]

@mhkabir The problem is that the transition from ARMING_STATE_INIT to ARMING_STATE_ARMED is invalid. The commander should have switched from ARMING_STATE_INIT to ARMING_STATE_STANDBY before
https://github.com/PX4/Firmware/blob/master/src/modules/commander/commander.cpp#L1820

[RomanBapst]

So you will need to put some printfs to see what's going on :)

[mhkabir]

Thanks, I'll have a look.

Looks like the issue is coupled to the sensor calibration of external mags. If I disable the UAVCAN mag and recalibrate, it's just fine.

[mhkabir]

@Tumbili I spent nearly two hours on this today without being any closer to finding out what the fail is.
The transition from INIT to standby is getting denied, but I still have absolutely no indication why. It seems to silently fail the transition. I've added printf's in all the suspect areas, but absolutely no luck.

Some help please :/

[mhkabir]

https://github.com/PX4/Firmware/blob/master/src/modules/commander/state_machine_helper.cpp#L250
Finally found it. That's where we fail. Oddly enough, the fail is never reported back anywhere. Any ideas @Tumbili ?

Kind of a impossible situation. No reason why it should do that. How does the sensor_feedback_provided even go to true??

How even :

mavlink_log_critical(mavlink_fd, "Silent fail s_feedb: %s", sensor_feedback_provided ? "true":"false");
            if (!sensor_feedback_provided || (new_arming_state == vehicle_status_s::ARMING_STATE_ARMED)) {
                mavlink_and_console_log_critical(mavlink_fd, "Not ready to fly: Sensors need inspection");
                sensor_feedback_provided = true;
            }

All I get is the first message, and sensor_feedback_provided is true. How is this even possible??

[LorenzMeier]

The issue we need to solve for is to report every sensor once, but not all the time. We might need an arming error bit mask and report only if the bit is not already set.

The bit mask should be re-set once a new link connects, which will automatically re-send the errors then.

[LorenzMeier]

A secondary issue is why its failing the check in the first place, of course.

[LorenzMeier]

Here, if this is set:
https://github.com/PX4/Firmware/blob/master/src/modules/commander/state_machine_helper.cpp#L245

then we would need here roughly this code:
https://github.com/PX4/Firmware/blob/master/src/modules/commander/state_machine_helper.cpp#L130

So in addition to fRunPrearm checks, a completely new if statement below that one:

/* re-run the pre-arm check as long as sensors are failing */
if (!status->condition_system_sensors_initialized && (new_arming_state == vehicle_status_s::ARMING_STATE_ARMED ||
new_arming_state == vehicle_status_s::ARMING_STATE_STANDBY)
                && status->hil_state == vehicle_status_s::HIL_STATE_OFF) {
            prearm_ret = prearm_check(status, mavlink_fd);
}

The issue you will run into is that it will spam you on boot. Which is why the vehicle_status needs to get that sensor fail bit mask and the pre-arm check should only report something via MAVLink if the failure bit is not already set. Then the whole reporting logic can be simplified based on that.

Let me know if you think you know what to do, else I'll have a go at this.

[mhkabir]

The bit mask should be re-set once a new link connects, which will automatically re-send the errors then.

@LorenzMeier Ah, I see.

I'd be glad if you had a try at fixing this. I spent nearly 3 hours on this. Kinda frustrated, sorry. I'll be happy to join in back tomorrow when I've got my head in order.

[mhkabir]

@LorenzMeier Woohoo! It works! No spamming, just like we wanted :) #3171

[20:07:01.651 - COMP:1] Info: new data link found #0
[20:07:01.878 - COMP:1] Critical: NOT ARMING: Flying with USB connected prohibited
[20:07:02.011 - COMP:1] Critical: Not ready to fly: Sensors need inspection

Still tracking down a thing or two, so PR is WIP

[LorenzMeier]

Thanks for fixing this!

[idbeja6]

@LorenzMeier I am also getting this error. I cant arm from the ground station GUI or from my remote. ESC is just beeping waiting and seems to be stuck in programming mode. LED safety switch is flashing quickly red and all sensors seem to be configured ok.

[LorenzMeier]

I presume you have 1) The beta firmware installed (if not, please install it instead of the default) and 2) USB NOT connected?