M99-endless-main-program test fails erratically

[zultron]

In #856, @jepler reported interp/m98m99/12-M99-endless-main-program test failures and disabled the test.

From the skip file, "test fails erratically with the last DISABLE line missing from out.motion-logger."

This issue is to track the intermittent problem.

[zultron]

I went through the last four weeks of CI build logs, both GH Actions and Travis CI, and couldn't find any runs that failed due to the interp/m98m99/12-M99-endless-main-program test. Maybe I missed it, but it was too painful to repeat.

@jepler Can you please provide a URL to the failed test?

[zultron]

@andypugh @jepler

We do have users that use M98/M99, including the "endless main program" feature described in Smid's "CNC Programming Handbook", which this test exercises. I'd like to keep the code and tests functional, if there are no objections.

[jepler]

Typical failure log: http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4983

--- expected.motion-logger	2020-05-17 02:15:05.352446987 -0600
+++ out.motion-logger	2020-05-17 02:30:33.148579492 -0600
@@ -199,4 +199,3 @@
 SET_LINE x=5.000000, y=-4.000000, z=0.000000, a=0.000000, b=0.000000, c=0.000000, u=0.000000, v=0.000000, w=0.000000, id=21, motion_type=2, vel=0.666667, ini_maxvel=4.000000, acc=1000.000000, turn=-1
 SET_SPINDLESYNC sync=0.000000, flags=0x00000000
 SPINDLE_OFF
-DISABLE
*** /home/buildslave/emc2-buildbot/wheezy-i386/rip-wheezy-i386/build/tests/interp/m98m99/12-M99-endless-main-program: FAIL: checkresult exited with 1

my suspicion is that it's a timing problem at shutdown. I didn't investigate fixes.

[jepler]

the buildbot is still the canonical source of truth about whether a build is "good", since we can't test the truly esoteric stuff (cough rtai cough) on newer CI systems

[zultron]

the buildbot is still the canonical source of truth about whether a build is "good", since we can't test the truly esoteric stuff (cough rtai cough) on newer CI systems

Ah, of course. It would be pretty neat to push buildbot status to GH PRs.

Thanks for the pointer to the failed build.

[zultron]

I'm wondering if this is related to (the newly-filed) #858.

Sample failures from the last month:
http://buildbot.linuxcnc.org/buildbot/builders/1510.rip-jessie-rtpreempt-i386/builds/3344
http://buildbot.linuxcnc.org/buildbot/builders/1520.rip-jessie-amd64/builds/3338
http://buildbot.linuxcnc.org/buildbot/builders/1660.rip-buster-python3/builds/45
http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4942
http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4983

Looks like we can expand the issue to include the motion-logger/mountaindew test:
http://buildbot.linuxcnc.org/buildbot/builders/1400.rip-wheezy-i386/builds/4855
http://buildbot.linuxcnc.org/buildbot/builders/1660.rip-buster-python3/builds/9