Run IPython tests with --colors=NoColor #14370

jdemeyer · 2013-03-27T22:39:01Z

On some systems:

Running doctests with ID 2013-03-27-15-38-22-527abcba.
Doctesting 2 files.
sage -t --long sage/misc/interpreter.py
**********************************************************************
File "sage/misc/interpreter.py", line 150, in sage.misc.interpreter.sage_prompt
Failed example:
    shell.run_cell('sage_prompt()')
Expected:
    u'sage'
Got:
    u'sage'
**********************************************************************
File "sage/misc/interpreter.py", line 566, in sage.misc.interpreter.interface_shell_embed
Failed example:
    shell.run_cell('List( [1..10], IsPrime )')
Expected:
    [ false, true, true, false, true, false, true, false, false, false ]
Got:
    [ false, true, true, false, true, false, true, false, false, false ]
**********************************************************************
2 items had failures:
   1 of   4 in sage.misc.interpreter.interface_shell_embed
   1 of   4 in sage.misc.interpreter.sage_prompt
    [107 tests, 2 failures, 3.3 s]
sage -t --long sage/misc/sage_extension.py
**********************************************************************
File "sage/misc/sage_extension.py", line 34, in sage.misc.sage_extension
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 39, in sage.misc.sage_extension
Failed example:
    shell.run_cell('%time 594.factor()')
Expected:
    CPU times: user ...
    Wall time: ...
    2 * 3^3 * 11
Got:
    CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
    Wall time: 0.00 s
    2 * 3^3 * 11
**********************************************************************
File "sage/misc/sage_extension.py", line 75, in sage.misc.sage_extension.SageMagics.runfile
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 99, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 103, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    3
**********************************************************************
File "sage/misc/sage_extension.py", line 106, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    []
**********************************************************************
File "sage/misc/sage_extension.py", line 127, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 131, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    3
**********************************************************************
File "sage/misc/sage_extension.py", line 134, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    []
**********************************************************************
File "sage/misc/sage_extension.py", line 210, in sage.misc.sage_extension.SagePlainTextFormatter
Failed example:
    shell.run_cell('a = identity_matrix(ZZ, 2); [a,a]')
Expected:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
Got:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
**********************************************************************
5 items had failures:
   2 of  12 in sage.misc.sage_extension
   3 of  14 in sage.misc.sage_extension.SageMagics.attach
   3 of  14 in sage.misc.sage_extension.SageMagics.pre_run_code_hook
   1 of   9 in sage.misc.sage_extension.SageMagics.runfile
   1 of   5 in sage.misc.sage_extension.SagePlainTextFormatter
    [55 tests, 10 failures, 2.3 s]
----------------------------------------------------------------------
sage -t --long sage/misc/interpreter.py  # 2 doctests failed
sage -t --long sage/misc/sage_extension.py  # 10 doctests failed
----------------------------------------------------------------------
Total time for all tests: 5.7 seconds
    cpu time: 0.8 seconds
    cumulative wall time: 5.6 seconds

Also, on OS X 10.8:

sage -t --long devel/sage/sage/doctest/test.py
**********************************************************************
File "devel/sage/sage/doctest/test.py", line 17, in sage.doctest.test
Failed example:
    subprocess.call(["sage", "-t", "longtime.rst"], cwd=tests_dir)  # long time
Expected:
    Running doctests...
    Doctesting 1 file.
    sage -t longtime.rst
    [0 tests, ...s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    ...
    0
Got:
    Running doctests with ID 2013-03-28-01-24-01-91891a00.
    Doctesting 1 file.
    [?1034hsage -t longtime.rst
        [0 tests, 0.0 s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    Total time for all tests: 0.1 seconds
        cpu time: 0.0 seconds
        cumulative wall time: 0.0 seconds
    0
**********************************************************************

Apply attachment: 14370_ipython_nocolors_v2.patch, attachment: 14370_run_doctests_fix.patch

Depends on #12415
Depends on #14331
Depends on #13278

Component: interfaces

Author: Jeroen Demeyer, Volker Braun

Reviewer: John Palmieri, David Roe

Merged: sage-5.9.beta3

Issue created by migration from https://trac.sagemath.org/ticket/14370

The text was updated successfully, but these errors were encountered:

vbraun · 2013-03-28T14:28:09Z

comment:4

The OSX issue is the same as #12263, either set TERM=dumb or ellipsize the smm escape sequence that may or may not be there.

It would be nice to run with #14290 to verify that the problem actually is color and not some other escape sequence.

jdemeyer · 2013-03-28T17:14:08Z

comment:5

Excellent diagnosis Volker!

jhpalmieri · 2013-03-28T17:34:27Z

comment:6

On OS X 10.8: I ran make ptest, and all tests passed. Immediately afterward, I ran

./sage -tp devel/sage/sage/misc/interpreter.py devel/sage/sage/misc/sage_extension.py

and I got the failures listed here. Does that make any sense?

vbraun · 2013-03-28T17:46:01Z

comment:7

Maybe make ptest changes TERM somewhere on OSX? Try with #14290 to actually see the escape sequences, otherwise we can just guess.

jhpalmieri · 2013-03-28T18:08:46Z

comment:8

With #14290:

$ ./sage -tp devel/sage/sage/misc/interpreter.py devel/sage/sage/misc/sage_extension.py
Running doctests with ID 2013-03-28-10-54-43-7fc9b4da.
Doctesting 2 files using 2 threads.
sage -t devel/sage/sage/misc/sage_extension.py
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 34, in sage.misc.sage_extension
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 39, in sage.misc.sage_extension
Failed example:
    shell.run_cell('%time 594.factor()')
Expected:
    CPU times: user ...
    Wall time: ...
    2 * 3^3 * 11
Got:
    CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
    Wall time: 0.00 s
    <CSI-0;31m><CSI-0m>2 * 3^3 * 11
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 75, in sage.misc.sage_extension.SageMagics.runfile
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 99, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 103, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    <CSI-0;31m><CSI-0m>3
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 106, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    <CSI-0;31m><CSI-0m>[]
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 127, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 131, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    <CSI-0;31m><CSI-0m>3
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 134, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    <CSI-0;31m><CSI-0m>[]
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 210, in sage.misc.sage_extension.SagePlainTextFormatter
Failed example:
    shell.run_cell('a = identity_matrix(ZZ, 2); [a,a]')
Expected:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
Got:
    <CSI-0;31m><CSI-0m>[
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
**********************************************************************
5 items had failures:
   2 of  12 in sage.misc.sage_extension
   3 of  14 in sage.misc.sage_extension.SageMagics.attach
   3 of  14 in sage.misc.sage_extension.SageMagics.pre_run_code_hook
   1 of   9 in sage.misc.sage_extension.SageMagics.runfile
   1 of   5 in sage.misc.sage_extension.SagePlainTextFormatter
    [55 tests, 10 failures, 2.6 s]
sage -t devel/sage/sage/misc/interpreter.py
**********************************************************************
File "devel/sage/sage/misc/interpreter.py", line 150, in sage.misc.interpreter.sage_prompt
Failed example:
    shell.run_cell('sage_prompt()')
Expected:
    u'sage'
Got:
    <CSI-0;31m><CSI-0m>u'sage'
**********************************************************************
File "devel/sage/sage/misc/interpreter.py", line 566, in sage.misc.interpreter.interface_shell_embed
Failed example:
    shell.run_cell('List( [1..10], IsPrime )')
Expected:
    [ false, true, true, false, true, false, true, false, false, false ]
Got:
    <CSI-0;31m><CSI-0m>[ false, true, true, false, true, false, true, false, false, false ]
**********************************************************************
2 items had failures:
   1 of   4 in sage.misc.interpreter.interface_shell_embed
   1 of   4 in sage.misc.interpreter.sage_prompt
    [107 tests, 2 failures, 8.8 s]
----------------------------------------------------------------------
sage -t devel/sage/sage/misc/sage_extension.py  # 10 doctests failed
sage -t devel/sage/sage/misc/interpreter.py  # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 8.9 seconds
    cpu time: 0.9 seconds
    cumulative wall time: 11.3 seconds

This command yields the same failures:

./sage -tp devel/sage/sage/misc/

This one passes all tests:

spkg/pipestatus "./sage -tp devel/sage/sage/misc 2>&1" "tee -a testmisc.log"

jhpalmieri · 2013-03-28T20:20:44Z

comment:10

The patch doesn't apply cleanly to 5.9.beta1. What are the dependencies?

jdemeyer · 2013-03-28T20:30:31Z

Dependencies: #12415, #14331, #13278

jhpalmieri · 2013-03-28T21:40:13Z

Reviewer: John Palmieri

jhpalmieri · 2013-03-28T21:40:13Z

comment:12

The code makes sense, and this seems to fix the problem. A minor fix:

diff --git a/sage/doctest/test.py b/sage/doctest/test.py
--- a/sage/doctest/test.py
+++ b/sage/doctest/test.py
@@ -13,7 +13,7 @@
     sage: tests_dir = os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests')
     sage: tests_env = dict(os.environ)
 
-Unset :envvar:TERM when running doctests, see :trac:`14370`::
+Unset :envvar:`TERM` when running doctests, see :trac:`14370`::
 
     sage: try:
     ....:     del tests_env['TERM']

Along these lines, is there a ticket to add files from the doctest directory to the reference manual?

jdemeyer · 2013-03-28T21:43:56Z

Attachment: 14370_ipython_nocolors.patch.gz

jdemeyer · 2013-03-28T21:47:49Z

comment:13

Replying to @jhpalmieri:

Along these lines, is there a ticket to add files from the doctest directory to the reference manual?

See #14376, you can go ahead and make a patch for it.

jdemeyer · 2013-04-01T12:39:04Z

comment:14

Attachment: 14370_ipython_nocolors_v2.patch.gz

jdemeyer · 2013-04-01T12:40:05Z

comment:15

I changed the patch slightly, I think this is a better solution. Note the change to doctest/forker.py, needs review.

vbraun · 2013-04-01T13:06:16Z

comment:17

I prefer v2, too. But run_doctests(sage.misc.interpreter) then still coughs up an escape code in shell.run_cell('List( [1..10], IsPrime )'), because that is not going through the doctest forker. I suppose run_doctests need not only disable color in the test shell but also the defaults if the doctest creates yet another shell.

vbraun · 2013-04-01T13:20:10Z

Attachment: 14370_run_doctests_fix.patch.gz

Initial patch

vbraun · 2013-04-01T13:21:36Z

comment:19

I've added the patch that does that. Positive review to Jeroen's patch.

roed314 · 2013-04-02T11:35:14Z

comment:20

Looks good to me.

jdemeyer · 2013-04-02T11:45:18Z

Merged: sage-5.9.beta3

jdemeyer · 2013-04-02T11:45:28Z

Changed reviewer from John Palmieri to John Palmieri, David Roe

jdemeyer · 2013-04-02T11:45:45Z

Changed author from Jeroen Demeyer to Jeroen Demeyer, Volker Braun

jdemeyer added this to the sage-5.9 milestone Mar 27, 2013

jdemeyer added c: interfaces labels Mar 27, 2013

jdemeyer assigned williamstein Mar 27, 2013

jdemeyer changed the title ~~Run IPython tests with --color=NoColors~~ Run IPython tests with --colors=NoColor Mar 27, 2013

jdemeyer added the s: needs review label Mar 27, 2013

This comment has been minimized.

Sign in to view

jdemeyer added s: needs work and removed s: needs review labels Mar 28, 2013

jdemeyer added s: needs review and removed s: needs work labels Mar 28, 2013

jhpalmieri added s: positive review and removed s: needs review labels Mar 28, 2013

jdemeyer added s: needs work and removed s: positive review labels Apr 1, 2013

jdemeyer removed the s: needs work label Apr 1, 2013

jdemeyer added the s: needs review label Apr 1, 2013

This comment has been minimized.

Sign in to view

roed314 added s: positive review and removed s: needs review labels Apr 2, 2013

jdemeyer removed the s: positive review label Apr 2, 2013

jdemeyer closed this as completed Apr 2, 2013

roed314 mentioned this issue Apr 4, 2013

Make sage -t --debug trigger on an incorrect doctest in addition to an unexpected error #13610

Closed

jdemeyer mentioned this issue Apr 1, 2013

Add doctest directory to the reference manual #14376

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run IPython tests with --colors=NoColor #14370

Run IPython tests with --colors=NoColor #14370

jdemeyer commented Mar 27, 2013

This comment has been minimized.

vbraun commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

vbraun commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jdemeyer commented Apr 1, 2013

jdemeyer commented Apr 1, 2013

This comment has been minimized.

vbraun commented Apr 1, 2013

vbraun commented Apr 1, 2013

This comment has been minimized.

vbraun commented Apr 1, 2013

roed314 commented Apr 2, 2013

jdemeyer commented Apr 2, 2013

jdemeyer commented Apr 2, 2013

jdemeyer commented Apr 2, 2013

Run IPython tests with --colors=NoColor #14370

Run IPython tests with --colors=NoColor #14370

Comments

jdemeyer commented Mar 27, 2013

This comment has been minimized.

vbraun commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

vbraun commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jhpalmieri commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jdemeyer commented Mar 28, 2013

jdemeyer commented Apr 1, 2013

jdemeyer commented Apr 1, 2013

This comment has been minimized.

vbraun commented Apr 1, 2013

vbraun commented Apr 1, 2013

This comment has been minimized.

vbraun commented Apr 1, 2013

roed314 commented Apr 2, 2013

jdemeyer commented Apr 2, 2013

jdemeyer commented Apr 2, 2013

jdemeyer commented Apr 2, 2013