Task Abort loop on joint soft limit

[SebKuzminsky]

In master after the JA merge, on a gantry machine. A misconfiguration put the soft limit of the Z axis and the soft limit of the joint of the Z axis in different places. Running in to the joint soft limit triggered a crazy Abort loop in Task:

    Exceeded POSITIVE soft limit (0.05000) on joint 2
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=1, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=88,type=EMC_TRAJ_SET_G5X}) : list_size=1, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=20,type=EMC_TRAJ_SET_ROTATION}) : list_size=2, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=24,type=EMC_TRAJ_SET_SPINDLESYNC}) : list_size=3, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_SPINDLE_OFF}) : list_size=4, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=5, line_number=0
    NML_INTERP_LIST(0x82f8398)::get(): {size=88, type=EMC_TRAJ_SET_G5X}, list_size=4
    emcTaskPlanLevel() returned 0
    Hint: switch to joint mode to jog off soft limit
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=5, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=88,type=EMC_TRAJ_SET_G5X}) : list_size=1, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=20,type=EMC_TRAJ_SET_ROTATION}) : list_size=2, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=24,type=EMC_TRAJ_SET_SPINDLESYNC}) : list_size=3, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_SPINDLE_OFF}) : list_size=4, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=5, line_number=0
    NML_INTERP_LIST(0x82f8398)::get(): {size=88, type=EMC_TRAJ_SET_G5X}, list_size=4
    emcTaskPlanLevel() returned 0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=5, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=88,type=EMC_TRAJ_SET_G5X}) : list_size=1, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=20,type=EMC_TRAJ_SET_ROTATION}) : list_size=2, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=24,type=EMC_TRAJ_SET_SPINDLESYNC}) : list_size=3, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_SPINDLE_OFF}) : list_size=4, line_number=0
    NML_INTERP_LIST(0x82f8398)::append(nml_msg_ptr{size=12,type=EMC_TASK_PLAN_SYNCH}) : list_size=5, line_number=0
    NML_INTERP_LIST(0x82f8398)::get(): {size=88, type=EMC_TRAJ_SET_G5X}, list_size=4
    emcTaskPlanLevel() returned 0

It never made any progress and I had to quit LinuxCNC to make it stop.

In another misconfiguration we hit the limit switch before the soft limit. In that situation it Aborted just once or twice, then chilled out.

[SebKuzminsky]

This bug can be reproduced with the sim/axis/gantry config by reducing [JOINT_3]MAX_LIMIT from 50 to 40, then homing and jogging towards the Y maximum.

[SebKuzminsky]

I dont think d118ec4 is right. Motion should abort on soft limit violation, but it no longer does:

This is with an un-modified sim/axis/axis, at the current tip of master (37e6fc4). I homed and jogged to near the X- soft limit, then programmed an MDI arc that violates that limit. Error messages were emitted, but the motion continued to completion.

[SebKuzminsky]

Part of the problem of this issue is that Axis switches to Teleop mode at a time when Motion is on a Joint soft limit. This causes the never-ending NML-issue-storm in Task. Inhibiting the switch to Teleop avoids the issue-storm.

Part of the fix should be to harden Task against this, either by making it gracefully handle the switch to Teleop, or by having it reject the command.

But another part of the fix should be to let the user recover from running into the Joint soft limit by jogging off it. Maybe on identity-kins machines we should allow Teleop-on-soft-limit, and allow Teleop jogging in the direction that leads back to the valid volume. But maybe on non-id-kins machines, Teleop outside the Joint limits might be a bad idea? Maybe we should force Free-mode jogging on non-id-kins machines?

I'm not sure what the right behaviors and restrictions here are, but i'm interested in having the conversation to come to an agreement.

[pkmcnc]

For identity-kins machines it's possible to avoid this situation completely. I see two options:

1. A config shouldn't load if axis limits are not within joint limits.
2. Ignore joint limits.

For non-identity kins it's more complicated. On the one hand, it's a configuration error. On the other hand, as working envelope is a cuboid, sometimes it's tempting to extend world limits to get larger envelope, meaning to avoid corners (in both linear and angular coordinates) where joint limits might bite. But technically it's still possible to reach those corners, so the limits will bite sooner or later.

What can be done in this situation? Stop, throw a warning "joint limit is exceeded" (that's what happens now) and give user a choice (probably via checkbutton "override soft limits") to get out of there by any means: jog into the limits in world or joint mode, or unhome to free mode, or re-home. When in joint mode, it would be perfect to highlight the joint value(s) in DRO.
Simpler way is force free mode. But on a machine with no home switches (manual homing) it might be painful to re-home from unusual position. And it still doesn't save the machine from breaking in case of wrong jogging.

BTW: I suggest to introduce cylindrical workspace limits for non-identity kins machines, it would be sufficient approximation in most cases.

[SebKuzminsky]

The idea of ignoring joint limits on id-kins machines feels strange after everyone worked so hard on the JA branch to create them, but maybe doing so simplifies things for (most) users.

If we instead go with load-time validation of joint and axis soft limits on id-kins machines, then the same check would be needed when the user sets these limits via HAL, using the interface in src/emc/ini/.

[pkmcnc]

Oh, the validation is pretty simple, and it can be skipped for inihal changes as long as the machine behaves more or less normally when the limits break.

But really, why would we need 2 sets of limits for one axis=joint? I think that joint limits can be safely and happily ignored for id-kins. This also simplifies INI file.

[SebKuzminsky]

How's this sound for a design:

id-kins machines

On id-kins machines the axis soft limits must be wholly contained within the joint soft limits. We can validate this at load-time, and when any soft limits are changed via inihal.

1. If any part of the joint soft limits is not specified, it defaults to the corresponding part of the axis soft limits.
2. If any part of the joint soft limits is more restrictive than the corresponding part of the axis soft limits it's a configuration error, and the config will fail to load (and emit a helpful error message).
3. When the user sets any soft limit via inihal, the check from 2 is applied. Open Question: How should failure be handled here? The inihal pins are inputs, and there's no standard error reporting mechanism in HAL.
4. In case of bugs, axis soft limits might still be violated (see for example Issue MDI of an arc that passes through the soft limit - overshoots soft limits by de-acceleration amount. #80).
5. Motion needs a way to communicate axis soft limit violation to Task and the UIs.
6. The UIs need a way to recover from the axis soft limit violation. Axis-mode jogging in the direction that leads back towards the axis soft limit interior seems like the best way to recover here. Other motions should probably be inhibited until the axis soft limit violation is resolved.

non-id-kins machines

On non-id-kins machines the axis soft limits may contain points that violate joint soft limits. For example, a robot arm might rotate around its base and reach below its base, but not operate in the volume actually occupied by its base:

1. The valid axis space is not a subset of the valid joint space. It is not a misconfiguration if the joint soft limits are more restrictive than the axis soft limits.
2. Therefore Interp and axis-mode jogs may accept motions that are valid in axis space but violate joint soft limits.
3. Motion needs a way to communicate this joint soft limit violation to Task and the UIs.
4. The UIs need a way to recover from the joint soft limit violation. Joint-mode jogging in the direction that leads back towards the joint soft limit interior seems like the best way to recover here. Other motions should probably be inhibited until the joint soft limit violation is resolved.
5. Axis soft limit violations can happen just like they can with id-kins machines (see above), and should be handled the same way. Open Question: What should we do if we're resolving an axis soft limit violation and we run in to a joint soft limit violation? Drop to joint mode and resolve that first, then resume with the axis soft limit violation recovert?

[pkmcnc]

Sounds pretty good.
I have some additional thoughts about non-id machines.

We should distinguish parallel (hexapod, delta) and serial (robot arm) manipulators.

Parallel robots usually have (more or less) cylindrical or cuboid workspace, axes limits are primary. Pretty much all we need for them is circular XY limits.

Serial robots have very complex shaped workspace. It's easy to see that it is completely described with joint limits. Hence, we don't need world limits for serial robots at all!
Therefore for serial robots joint limits should be primary, axes limits can be ignored. It probably requres a bit different protocol to handle joint limits.

Also, checking every line of G-code for joint limits is difficult and (probably) not always possible. But it's not nesessary! I believe that cartesian G-code can be verified for joint limits with full simulation. Say, live plot combined with inverse kins (just a guess).

[SebKuzminsky]

I think non-cuboid axis soft limits are an interesting idea, but outside the scope of this particular issue.

Axis soft limits should probably be specified by the kinematics component, but this introduces a huge change in the architecture of LinuxCNC since kins is currently part of Motion (which is realtime), and axis soft limits need to be available to Interp in Task (which is non-realtime). Feel free to open another issue for this idea, but let's keep discussion in this issue focused on un-breaking the good old cuboid axis soft limits that we currently have, that broke with the recent merge of JA.

[propcoder]

Serial robots have very complex shaped workspace. It's easy to see that it is completely described with joint limits. Hence, we don't need world limits for serial robots at all!

I don't fully agree with that. World limits or other temporary workspace limiting capability would be needed for example, when serial robot has to work in tight cabinet.