save timeit information with sage -t -timeit

[ncalexan]

There are several parts to this patch.

The first is an update to sage.misc.sage_timeit that generalizes the interface to return an object that prints itself as a timing string rather than the string itself. The advantage is that the information does not need to be parsed from the string later. This patch is a requirement of the later ones but is conceptually independent.

The second adds a file ncadoctest.py to scripts that is a slightly modified version of Python's doctest.py file. It is easier to subclass the various classes with this version.

The third uses ncadoctest.py to subclass the doctest architecture and updates sage-doctest to use these updated classes.

Component: doctest coverage

Keywords: testing doctest timing timeit profile regression, editor_mhansen

Issue created by migration from https://trac.sagemath.org/ticket/3476

[ncalexan]

Attachment: 3476-ncalexan-sage-scripts-doctest-1.patch.gz

[ncalexan]

comment:1

Attachment: 3476-ncalexan-sage-timeit-1.patch.gz

The attachments came in the wrong order -- the one with sage-scripts applies to sage/local/bin.

[ncalexan]

comment:2

Attachment: diff-python-doctest-to-ncadoctest.gz

diff-python-doctest-to-ncadoctest records the changes from upstream Python doctest.py to ncadoctest.py for future reference.

[sagetrac-mabshoff]

Changed keywords from testing doctest timing timeit profile regression to testing doctest timing timeit profile regression, editor_mabshoff

[garyfurnish]

Changed keywords from testing doctest timing timeit profile regression, editor_mabshoff to testing doctest timing timeit profile regression, editor_mabshoff, editor_gfurnish

[rlmill]

comment:5

This should be applied after #3982.

[rlmill]

Changed keywords from testing doctest timing timeit profile regression, editor_mabshoff, editor_gfurnish to testing doctest timing timeit profile regression, editor_mhansen

[mwhansen]

comment:7

Since my one main concern at #3982 is taken care of, I think this can go in.

[sagetrac-mabshoff]

comment:8

There is a reject apllying Nick's first patch:

--- sage-doctest
+++ sage-doctest
@@ -218,8 +241,8 @@ def extract_doc(file_name, module):
             doc = doc_preparse(F[i:j+3])
         except SyntaxError:
             doc = F[i:j+3]
-        if len(doc):
-            doc = '""">>> set_random_seed(0L)\n\n' + doc[3:]
+#         if len(doc):
+#             doc = '""">>> print "YYY"; print random() # ; set_random_seed(0L)\n\n' + doc[3:]
         s += "\tr"+ doc + "\n\n"
         F = F[j+3:]

I am attempting to merge this manually.

Cheers,

Michael

[sagetrac-mabshoff]

comment:9

Ok, the reject seems to happen due to merging the warning patch into sage-doctest.

Cheers,

Michael

[sagetrac-mabshoff]

comment:10

If I merge the patch without the troublesome hunk I see 4 doctests related to the random framework and timeit itself fail:

	sage -t -long devel/sage/sage/misc/sage_timeit_class.pyx # 7 doctests failed
	sage -t -long devel/sage/sage/misc/sage_timeit.py # 2 doctests failed
	sage -t -long devel/sage/sage/misc/prandom.py # 2 doctests failed
	sage -t -long devel/sage/sage/misc/randstate.pyx # 6 doctests failed

Oh well, life sucks :)

Cheers,

Michael

[mwhansen]

Attachment: trac_3476.patch.gz

Attachment: trac_3476-scripts.patch.gz

[mwhansen]

comment:12

I've added two updated patches which fix the issue. The problem was that the 'timeit' in test.globs was set to "False" from the options in sage-doctest instead of being the actual timeit function from Sage. Thus, you'd only hit the problem with doctests that used timeit.

[sagetrac-mabshoff]

comment:13

For "sage -sdist" to work we need to copy ncadoctest.py, sagedoctest.py in sage-make_devel_packages after

  cp -p $SAGE_ROOT/local/bin/SbuildHack.pm $SCRIPTS/

I will take care of this once the patch passes doctests.

Cheers,

Michael

[sagetrac-mabshoff]

comment:14

Hi Mike,

there is one tiny easy to fix doctest issue left:

sage -t -long devel/sage/sage/misc/sage_timeit.py           
**********************************************************************
File "/scratch/mabshoff/release-cycle/sage-3.2.rc0/devel/sage/sage/misc/sage_timeit.py", line 48:
    sage: sage_timeit("a = 2\nb=131\nfactor(a^b-1)", globals(), number=10)
Expected:
    '10 loops, best of 3: ... per loop'
Got:
    10 loops, best of 3: 18.4 ms per loop
**********************************************************************

I will fix this via a followup patch.

Cheers,

Michael

[sagetrac-mabshoff]

comment:15

Merged in Sage 3.2.rc0