eyelid motor hardware fault

[amaroyo]

Robot name 🤖

iCubWaterloo01

Request/Failure description

Hardware fault eyelid motor - I checked that the motor is engaging the eyelid, and starting from middle position.

Detailed context

Hi @violadelbono,

It's me again... I know we had problems before with the eyelid but we managed to reach a dirty fix. Now I was just testing them, but didn't manage to resolve the hardware fault.

The only difference from before and now, is:
-we upgraded the laptop from ubuntu 18 to 20.
-upgraded both the laptop and iCub-head from 2020.08 (I guess?) to the 2021.01
-still need to upgrade the ubuntu of icub's head but we will do after the iCub course.

From the yarprobot interface, the only error I was reading was the following (but not sure if it's related):
[ERROR] from BOARD 10.0.1.22 (face-eb22-j0), src LOCAL, adr 0, time 300s 621m 52u: (code 0x02000010, par16 0x0000 par64 0x0000000000000000) -> MC: hard limit reached.

I also tried opening the canLoader but didn't managed... is it deprecated?! What do you use instead?

Any suggestions?

Thanks,
Alex.

Additional context

No response

[violadelbono]

Ciao @amaroyo

The error you reported shows that the joint went outside of its software "safe" boundaries, meaning that the eyelids opened too much (I guess) and thus in this state it is not controllable. Some questions:

• Did it happen when you were moving the eyelids with the motorgui?
• If yes, did you reach the maximum range of motion? It could be a calibration problem.
• I suggest you also to place the eyelids in a middle position, restart the robot, and try again to move them to see if it repeats.

CANloader is deprecated, now we use Firmwareupdater to manage the boards. However, this doesn't seem related.

Let me know,
Viola

[amaroyo]

Ciao @violadelbono!

No, the eyelids are already in hardware fault when I start the robot. There's not a calibration phase of the eyelids.

The first time I started the robot, the eyelids were fully open; the second time I started it, they were in the middle position but still no result.

I thought a board that controls the motor may not be responsive... Do you think it could be from the sw update?

Thanks,
Alex.

[violadelbono]

@amaroyo You can check the board status in this way:

• run Firmwareupdater on a terminal
• Select ETH and click on discover
• check the FW version of the board 10.0.1.22 (if you could take a screenshot and post it here would be great)

I don't know if it could be due to the update, but since you updated icub-head I don't exclude it.

[violadelbono]

Also some documentation related to Firmware is here.

[amaroyo]

Ciao @violadelbono,

My fault, I will give you a better update. There are more errors on the yarprobotinterface:

[ERROR]  from BOARD 10.0.1.22 (face-eb22-j0), src LOCAL, adr 0, time 636s 202m 727u: (code 0x02000010, par16 0x0000 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

[ERROR]  from BOARD 10.0.1.25 (left_arm-eb25-j8_11), src LOCAL, adr 0, time 637s 906m 232u: (code 0x02000010, par16 0x0002 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

[ERROR]  from BOARD 10.0.1.26 (left_arm-eb26-j12_15), src LOCAL, adr 0, time 638s 347m 34u: (code 0x02000010, par16 0x0003 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

[ERROR] xsensmt: Sensor timeout, no sensor measurement received in the last 0.205188 seconds.

[ERROR]  from BOARD 10.0.1.28 (right_arm-eb28-j8_11), src LOCAL, adr 0, time 639s 787m 266u: (code 0x02000010, par16 0x0002 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

[ERROR]  from BOARD 10.0.1.29 (right_arm-eb29-j12_15), src LOCAL, adr 0, time 640s 224m 903u: (code 0x02000010, par16 0x0003 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

[ERROR] Left_Arm_Calibrator : detected an hardware fault on joint  9  before starting the calibration procedure!
[ERROR] Left_Arm_Calibrator : detected an hardware fault on joint  10  before starting the calibration procedure!
[ERROR] Left_Arm_Calibrator : detected an hardware fault on joint  11  before starting the calibration procedure!
[ERROR] Right_Arm_Calibrator : detected an hardware fault on joint  12  before starting the calibration procedure!
[ERROR] FaceV3_Calibrator : detected an hardware fault on joint  0  before starting the calibration procedure!
[ERROR] Right_Arm_Calibrator : detected an hardware fault on joint  13  before starting the calibration procedure!
[ERROR] Right_Arm_Calibrator : detected an hardware fault on joint  14  before starting the calibration procedure!
[ERROR] Right_Arm_Calibrator : detected an hardware fault on joint  15  before starting the calibration procedure!
[ERROR] unable to complete calibration: joint 0 in 'hw_fault status' inside doneRaw() function

This only happens at the beginning, when I launch the yarprobotinterface. Once the robot calibrated, the only error message that keeps repeating itself is the one of the board:

[ERROR]  from BOARD 10.0.1.22 (face-eb22-j0), src LOCAL, adr 0, time 636s 202m 727u: (code 0x02000010, par16 0x0000 par64 0x0000000000000000) -> MC: hard limit reached. The joint position is outside its hardware boundaries. par16 = ID of joint. + .

So, the boards 25,26,28,29 are the arms... but I can move them! The 22 is the Skin torso, right? But the skin works as well! :D

Still, when checking in yarpmotorgui, only the eyelid joint is in hardware fault..... So, the issue we have with the eyelid is not appearing in the yarprobotinterface, right? And the other errors we see in the yaprobotinterface, are false positives?

I am adding the Firmwareupdater screenshots of all those boards below.

Maybe it's going to be easier to just set up a call and go through it.. what do you think?

Cheers,
Alex.

[violadelbono]

The 22 is the Skin torso, right? But the skin works as well! :D
So, the issue we have with the eyelid is not appearing in the yarprobotinterface, right?

Actually the board 22 has the CAN channel connected to the skin torso, but it also controls the eyelids motor and encoder. The error related to board 22 that we see, regards the eyelids.

The FW of all the boards looks fine to me. It seems that the fingers were in HW fault at startup (joints 9 to 15), not the arms.

We can setup a call! Next week I will be busy from Mon to Wed. So I propose either tomorrow or Thursday 13th

What do you prefer?

Best,
Viola

[amaroyo]

Let's do tomorrow, Friday the 7th... I will send you an email!

[violadelbono]

The issue was solved by recalibrating the eyelids. I think we can close it.