Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timing instability in doctest reporter #16109

Closed
vbraun opened this issue Apr 9, 2014 · 11 comments
Closed

Timing instability in doctest reporter #16109

vbraun opened this issue Apr 9, 2014 · 11 comments

Comments

@vbraun
Copy link
Member

vbraun commented Apr 9, 2014

Random doctest failure on the buildbot:

sage -t --long src/sage/doctest/reporting.py
**********************************************************************
File "src/sage/doctest/reporting.py", line 265, in sage.doctest.reporting.DocTestReporter.report
Failed example:
    DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D), "Good tests")
Expected:
        [... tests, 0.00 s]
Got:
        [103 tests, 0.01 s]
**********************************************************************
File "src/sage/doctest/reporting.py", line 275, in sage.doctest.reporting.DocTestReporter.report
Failed example:
    DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D), "Doctest output including the failure...")
Expected:
        [... tests, 1 failure, 0.00 s]
Got:
        [103 tests, 1 failure, 0.01 s]
**********************************************************************
File "src/sage/doctest/reporting.py", line 289, in sage.doctest.reporting.DocTestReporter.report
Failed example:
    DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D), "Good tests")
Expected:
        1 unlabeled test not run
        4 long tests not run
        5 magma tests not run
        2 other tests skipped
        [... tests, 0.00 s]
Got:
        1 unlabeled test not run
        4 long tests not run
        5 magma tests not run
        2 other tests skipped
        [103 tests, 0.01 s]
**********************************************************************

obviously we shouldn't doctest that the time is really 0.00s

Component: doctest framework

Keywords: days57

Author: Volker Braun

Branch/Commit: 2d73de6

Reviewer: Jeroen Demeyer

Issue created by migration from https://trac.sagemath.org/ticket/16109

@vbraun vbraun added this to the sage-6.2 milestone Apr 9, 2014
@vbraun
Copy link
Member Author

vbraun commented Apr 9, 2014

Author: Volker Braun

@vbraun

This comment has been minimized.

@vbraun
Copy link
Member Author

vbraun commented Apr 9, 2014

@vbraun
Copy link
Member Author

vbraun commented Apr 9, 2014

Changed keywords from none to days57

@vbraun
Copy link
Member Author

vbraun commented Apr 9, 2014

Commit: 2d73de6

@vbraun
Copy link
Member Author

vbraun commented Apr 9, 2014

New commits:

2d73de6do not test that elapsed time is 0.00s

@jdemeyer
Copy link

comment:4

Could you please tell me which buildbot this is? I am asking because I have never seen this error before and I want to make sure that doctesting didn't actually become slower.

@jdemeyer
Copy link

Reviewer: Jeroen Demeyer

@jdemeyer
Copy link

comment:5

Obviously, the patch makes sense, but it's strange that this error suddenly appears.

@vbraun
Copy link
Member Author

vbraun commented Apr 11, 2014

comment:6

I've seen it once so far. I don't remember which buildbot but one of the UW machines. I think there were a lot of background jobs running.

@vbraun
Copy link
Member Author

vbraun commented Apr 14, 2014

Changed branch from u/vbraun/timing_instability_in_doctest_reporter to 2d73de6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants