Certain tests still fail randomly in CI

[prjemian]

Until it is understood why certain tests fail have random failures in CI, mark them with @pytest.mark.local.

Originally posted by @prjemian in #747 (comment)

• random CI failures, unidentified reason #696

[prjemian]

Since the CI workflow takes so long to complete (cycle times can be >10m), consider diagnosing this in a separate Repo.

[prjemian]

CI is passing now, skipping all tests marked pytest.mark.local. Need to enable these tests in CI to observe if marked tests continue to fail.

[prjemian]

Failed again, locally:

apstools/apstools/devices/tests/test_positioner_soft_done.py

Line 275 in 44d73bc

assert abs(rb - sp) <= tol, f"setpoint={sp}, readback={rb}, tolerance={tol}"

[prjemian]

Replace failing assert per stackoverflow. (Function to determine if two numbers are nearly equal when rounded to n significant decimal digits)

[prjemian]

We've got a tolerance, not the number of sig figs. Thus, use math.isclose() over numpy.testing.assert_approx_equal().

[prjemian]

Again! Different place.

        assert round(pos.readback.get(use_monitor=False), 2) == known_start_position
>       assert pos.inposition, f"target={pos.setpoint.get()}, readback={pos.position}"
E       AssertionError: target=-1.0, readback=-1.0
E       assert False
E        +  where False = PVPositionerSoftDoneWithStop(prefix='gp:gp:', name='pos', settle_time=0.0, timeout=None, read_attrs=['readback', 'setpoint'], configuration_attrs=['done', 'tolerance', 'target'], limits=None, egu='').inposition

/home/runner/work/apstools/apstools/apstools/devices/tests/test_positioner_soft_done.py:321: AssertionError

[prjemian]

Another:

>       assert math.isclose(rb, sp, abs_tol=positioner.actual_tolerance)
E       AssertionError: assert False
E        +  where False = <built-in function isclose>(0.0, -1.0, abs_tol=1e-05)
E        +    where <built-in function isclose> = math.isclose
E        +    and   1e-05 = PVPositionerSoftDoneWithStop(prefix='', name='calcpos', settle_time=0.0, timeout=None, read_attrs=['readback', 'setpoint'], configuration_attrs=['done', 'tolerance', 'target'], limits=None, egu='').actual_tolerance

/home/runner/work/apstools/apstools/apstools/devices/tests/test_positioner_soft_done.py:275: AssertionError

[prjemian]

The setpoint is new but the readback suggests the positioner has not moved. Perhaps the problem lies in the implementation code, not the test code?

[prjemian]

Replace the three tests. For now, restore the @pytest.mark.local decorator.