timeouts in doctests on OS X 10.9

[jhpalmieri]

There are frequent timeouts when doctesting on OS X with pexpect interfaces, in particular Singular. For example:

$ ./sage -tp src/doc/en/constructions

...
----------------------------------------------------------------------
sage -t src/doc/en/constructions/algebraic_geometry.rst  # Timed out
sage -t src/doc/en/constructions/polynomials.rst  # Timed out
----------------------------------------------------------------------

These became much more frequent by the changes in the Singular/pexpect interface from #15631 on two buildbots:

• OSX 10.9 (Oxford)
• Ubuntu 8.04 (UW)
• This happens mostly during parallel doctesting, and disappers when the system load is low
  Other build bots do not seem to be affected, so its a likely pty kernel bug.

CC: @vbraun

Component: interfaces

Author: Volker Braun

Branch/Commit: 2928b9a

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/16474

[dimpase]

comment:2

even after reverting the corresponding commit from #15631 I still get lots of doctest timeouts.

[jhpalmieri]

comment:3

I'm not suggesting this as a fix (because I don't know anything about the pexpect interface), but if I make this one change:

diff --git a/src/sage/interfaces/singular.py b/src/sage/interfaces/singular.py
index 576fcc7..97ed05d 100644
--- a/src/sage/interfaces/singular.py
+++ b/src/sage/interfaces/singular.py
@@ -378,7 +378,7 @@ class Singular(Expect):
         """
         prompt = '> '
         Expect.__init__(self,
-                        terminal_echo=False,
+                        terminal_echo=True,
                         name = 'singular',
                         prompt = prompt,
                         command = "Singular -t --ticks-per-sec 1000", #no tty and fine grained cputime()

then I don't get the timeouts any more. (I get other doctest failures because of this, but no timeouts.)

[dimpase]

comment:4

in case, note #16260#comment:62

[vbraun]

comment:5

I've debugged the issue on OSX 10.9 and when the hang happens

• Singular does print its header and prompt (captured with dtrace)
• Python never sees any output from the subprocess, the first select() in pexpect.read_nonblocking hangs forever.

One way to reduce the likelihood of the hang is to insert a sleep before first reading from the subproces (in expect._start)

[vbraun]

comment:6

Sleeping 100ms when starting Singular reduces the failure rate in sage -t --long src/sage/interfaces/magma.py from about 1/10 to 1/1000 fwiw.

[vbraun]

Branch: u/vbraun/timeouts_in_doctests_on_os_x_10_9

[vbraun]

comment:8

The branch attached is obviously a no-op, and just moves the setecho(0) further down after we read the first prompt. But it seems to improve things in my limited testing.

---

New commits:

2928b9a

Fix Singular pexpect timeouts on some platforms

[vbraun]

Author: Volker Braun

[vbraun]

Commit: 2928b9a

[vbraun]

comment:9

This seems to fix it, I haven't seen another failure in sage -t --long src/sage/interfaces/magma.py

[jhpalmieri]

comment:10

Oh, very nice. This fixes it on two OS X 10.9 machines.

[jhpalmieri]

Reviewer: John Palmieri

[vbraun]

Changed branch from u/vbraun/timeouts_in_doctests_on_os_x_10_9 to 2928b9a