TestSuite improvements

[hivert]

• Changes "... done" to ". . . pass" in the output of TestSuite().run(verbose = True) to avoid unintentional matches
• Fix the doctests accordingly.
• Adds skip option; use it in sage/combinat/sf/jack.py and orthotriang.py
• In case of failure, execute the following tests after printing out a traceback, and write a summary at the end
• Only use verbose=True in the doctests when useful (category examples)

Rationale for the ...: when testing something in verbose mode the typical output of sage is:

   sage: P = Sets().example("inherits")
   sage: TestSuite(P).run(verbose=True)
   running ._test_an_element() ... done
   running ._test_element_pickling() ... done
   running ._test_not_implemented_methods() ... done
   running ._test_pickling() ... done
   running ._test_some_elements() ... done

And there is some risks that the "..." match something they should'nt I change them to ". . ."

See discussion on sage-devel

CC: @sagetrac-sage-combinat

Component: doctest coverage

Keywords: TestSuite

Author: Florent Hivert, Nicolas M. Thiéry

Reviewer: Nicolas M. Thiéry, Florent Hivert

Merged: sage-4.3.alpha0

Issue created by migration from https://trac.sagemath.org/ticket/7478

[nthiery]

Reviewer: Nicolas M. Thiéry, Florent Hivert

[nthiery]

Changed author from Florent Hivert to Florent Hivert, Nicolas M. Thiéry

[mwhansen]

comment:4

There is a spurious "only" to "o.." change in sets_cat.py.

[nthiery]

comment:5

Replying to @mwhansen:

There is a spurious "only" to "o.." change in sets_cat.py.

Good spot :-)

I had already found a couple, and thought I had done a systematic search.

Fixed in the newly uploaded patch.

[nthiery]

comment:6

Updated patch fix a doctest I had missed.

[hivert]

comment:7

There are still some minor problems (missing ``...`` in the documentation of testsuite. Aside those everything is ok for me. Nicolas can you review
trac_7478_testsuite_dots-fh-review.patch
on the queue. Then either me or you fold, reupload and set positive review.

...trying to ping you on irc for synchro.

[nthiery]

comment:8

Attachment: trac_7478_testsuite_dots-fh.patch.gz

Replying to @hivert:

There are still some minor problems (missing ``...`` in the documentation of testsuite. Aside those everything is ok for me. Nicolas can you review
trac_7478_testsuite_dots-fh-review.patch
on the queue. Then either me or you fold, reupload and set positive review.

...trying to ping you on irc for synchro.

Review patch is good. Folded and uploaded. positive review.

[mwhansen]

Merged: sage-4.3.alpha0

[qed777]

comment:10

I just "discovered" sage.misc.sage_unittest. In case it's of wider use: At #7390 there's a unittest HTML report generator that may make it into SageNB.