firmware-installation fails on a Jasper drive: the step to INIT counts a broadcast INIT from a drive that restarted and lost its station address

[oemamgholi]

User story

As an engineer updating a Jasper drive with the firmware-installation procedure,
I want the procedure to follow the drive through the restart its bootloader makes on the way out of BOOT,
so that the installation ends in PRE-OP on the new firmware, as it does on a SOMANET drive and on an Integro, and as it does with Motion Master v5.

Context

What the Jasper bootloader does. When an application (app_*) or a PRU firmware (com_*) was committed in BOOT, the bootloader warm-resets the device on BOOT → INIT, so that the new image starts from a cold boot. The reset re-initialises the EtherCAT slave controller: the link drops, and the station address is cleared. About 1.9 s after the INIT request the device answers again, in INIT; most of that time is the Ethernet link coming back up. The next INIT → PRE-OP starts the new application. A BOOT session that committed nothing leaves BOOT without a reset.

What happens. 6.0.0-alpha.115 on Linux with the SOEM driver, a Jasper NGC drive (4201-01) alone on the bus. The code involved is unchanged in 6.0.0-alpha.117. The procedure writes the package, then fails in its final-state step:

the firmware was written; the device did not reach PRE-OP: could not reach INIT: the device reported UNKNOWN rather than INIT

The frames that came back to the master's NIC, from the INIT request on:

Time	Frame	What it means
0	FPWR 0x0120 = 0x01 at 0x1001	INIT requested. The last AL status read was BOOT.
0 to 1.94 s	none	The device restarts.
1.94 s	BRD 0x0130 → INIT, wkc 1	The poll in transitionToState counts INIT as reached.
1.94 s	FPRD 0x0130 at 0x1001 → wkc 0; APRD → INIT	readStates finds the address lost; the state reads as 0, "UNKNOWN".

The drive itself is fine: it is in INIT on the new firmware, only without its station address. A POST /api/devices/state to PRE-OP right afterwards reaches PRE-OP in 2.6 s, with "it had reset and lost its station address; re-assigned 0x1001 at its wire position" and "reprogramming mailbox sync managers after its restart" in the log.

Why the existing restart handling does not catch it. SoemFieldbusDriver::transitionToState admits a slave whose address was lost when mayRestart(pos), and gives the address back in its resend block, which runs every resendInterval (2 s) for the slaves still pending. The poll reads the state with ecx_readstate. When every slave on the bus reports the same state, SOEM takes it from one broadcast read of 0x0130, which a device answers without a station address. A device that comes back from its restart already in INIT therefore reaches the target at the first poll after its link is up, before any resend, so its address is never restored. DeviceManager::transitionToState then reads the state back with readStates, whose probeReachability finds the address lost, and transitionTo sees state 0.

No reset time avoids this for a device that restarts during the step to INIT and comes back in INIT. The devices the code describes pass because their restart comes after the read-back:

• SOMANET restarts on the step to PRE-OP. After its restart it is in INIT, not PRE-OP, so it stays pending and the resend restores it.
• Integro with a new com binary reports INIT and resets its netX 500 ms later (HIL_FIRMWARE_RESET_REQ, ulTimeToReset = 500, in Boot_RebootForFwUpdate, somanet_software). The 100 ms poll and the read-back see INIT before the reset, which then lands in the step to PRE-OP.
• From reading the code, not tested: an Integro whose INIT is missed, the case 6.0.0-alpha.94 (An Integro install stops at the BOOT exit, although the firmware was written #115) gave the INIT step its address recovery for, fails the same way on a bus of one device, because it comes back reporting INIT and the broadcast poll accepts it before the resend. On a bus whose devices report different states, ecx_readstate reads each device at its address instead, the restarted device reads 0, stays pending, and the resend restores it.

Motion Master v5 (OBLAC, motion_master 5.8.7) installs the same packages on the same drive and bootloader. It reports the installation finished after the FoE writes, then deinitialises the bus, accepts INIT from a broadcast read, and scans the bus again, which assigns the station address anew.

Proposed change

In SoemFieldbusDriver::transitionToState, before counting a slave that mayRestart as having reached the target, read its state at its configured address. When that read finds the address lost (probeReachability → AddressLost), restore the address at once (restoreStationAddress, and updateMailboxSyncManagers when the target is PRE-OP), read the state at the address, and judge that instead of waiting for the next resend. A slave that answers at its address takes the path it takes today, so SOMANET drives and Integros keep their current behaviour.

A smaller alternative is in leaveBoot: when the step to INIT ends with the device address-lost, restore the address and read the state again before failing. The driver change is preferable: it covers both steps out of BOOT, and also the Integro whose INIT was missed.

Acceptance criteria

• firmware-installation of a package with app_*.bin and com_*.bin on a Jasper drive with its current bootloader, alone on the bus, succeeds and ends in PRE-OP on the new firmware.
• The same with finalState INIT ends in INIT, with the drive answering at its station address.
• A slave that restarts during a step out of BOOT and comes back in the target state counts as reached only after it answers at its station address.
• SOMANET drives install as today: the restart on the step to PRE-OP is still recovered.
• An Integro with a new com binary installs as today.
• Unit tests: a slave that answers the broadcast state read but not its configured address until the address is written, for the step to INIT and the step to PRE-OP.
• Hardware-verified on a Jasper drive. We can run a pre-release build on our bench, including our hardware test of the procedure.

Out of scope

• The default skipFiles names the ESI SOMANET_CiA_402.xml.zip; Jasper packages carry SOMANET_CiA_402.xml, so it is written to the drive. That belongs to the Jasper package format, not to this issue.
• Changing when the Jasper bootloader resets.

References

At v6.0.0-alpha.117:

• libs/comm/soem_fieldbus_driver.cc: transitionToState (mayRestart at L2013, the reached check at L2169, the resend block at L2202–2218), restoreStationAddress (L1909), probeReachability (L77), readStates (L813)
• libs/node/device_manager.cc: DeviceManager::transitionToState, the driver call and the read-back (L1188–1192)
• libs/node/somanet_procedures.cc: transitionTo (L676), leaveBoot (L730), kBootExitTimeout (L602)
• CHANGELOG 6.0.0-alpha.94, "The step from BOOT to INIT waits for the restart of an Integro" (An Integro install stops at the BOOT exit, although the firmware was written #115)
• somanet_software, arm_application/Components/Boot/Sources/AppECS_Functions_Boot.c: Boot_Handle_ESM_AL_Control_Changed_Ind, Boot_RebootForFwUpdate

[markosankovic]

This could be related to #175

I'll investigate both issues.

[markosankovic]

@oemamgholi resolved in #190