Doctest 2 to 3 #15

janwijbrand · 2016-09-29T16:27:27Z

Introduce optionflag EXCEPTION_2TO3 to normalize exception class names in traceback output. In Python 3 they are displayed as the full dotted name. In Python 2 they are displayed as "just" the class name. When running doctests in Python 3, the optionflag will not have any effect, however when running the same test in Python 2, the segments in the full dotted name leading up to the class name are stripped away from the "expected" string.

…names in traceback output.

…raceback output.

mgedmin · 2016-09-30T06:52:22Z

src/zope/testing/renormalizing.py

@@ -39,6 +53,10 @@ def check_output(self, want, got, optionflags):
            want = transformer(want)
            got = transformer(got)

+        if sys.version < '3':


I would prefer if sys.version_info[0] < 3:

We'll update this

mgedmin · 2016-09-30T06:52:58Z

src/zope/testing/renormalizing.py

@@ -62,6 +80,12 @@ def output_difference(self, example, got, optionflags):

        # temporarily hack example with normalized want:
        example.want = want
+
+        if sys.version < '3':


Here too.

Who knows, there may be a Python 10 at some point.

We'll update this

mgedmin · 2016-09-30T06:56:04Z

src/zope/testing/renormalizing.py

+
+
+def strip_dottedname_from_traceback(string):
+    # We might want to confirm more strictly were dealing with a traceback.


I'd prefer that. See doctest.py for the sigh private regexp. I think a simpler version that checks just the first line would suffice.

We'll look in to this.

mgedmin · 2016-09-30T06:56:54Z

src/zope/testing/renormalizing.txt

+    ... else:
+    ...     expected = False
+    >>> result == expected
+    True


Ugh.

Maybe just do it always, on Python 2 and Python 3? Stripping a non-existent prefix is a no-op anyway.

We're not sure we understand what you mean.

We took the conscious decision to use Python 3 as the "baseline" so to say. Meaning, when running the tests in Python 3 we do not touch the output. This means, when fixing tests, you "move forward" so to say instead of sticking to Python 2.

mgedmin · 2016-09-30T06:58:45Z

src/zope/testing/renormalizing.txt

+    ...     HINT:
+    ...       You seem to test traceback output.
+    ...       The optionflag EXCEPTION_2TO3 is set.
+    ...       Do you use the full dotted name for the exception class name?


This hint is not helpful. Is it saying that I should use the full dotted name, or that I shouldn't?

I'm starting to think that instead of # doctest: +EXCEPTION2TO3 I want #doctest: +IGNORE_EXCEPTION_MODULE, and then do the stripping on both the want and the got texts, irrespective of the current Python version.

We'll try to update the hint to more helpful.

We'll update the name of the flag to IGNORE_EXCEPTION_MODULE_IN_PYTHON2 due to the aforementioned "conscious decision".

That's uncomfortably long :(

mgedmin · 2016-09-30T06:59:16Z

src/zope/testing/test_renormalizing.py

+foo.bar.FooBarError: requires at least one argument."""
+        expected = """\
+Traceback (most recent call last):
+FooBarError: requires at least one argument."""


Please use textwrap.dedent() here and in all the other tests in this file, to make this readable.

Will look in to that.

janwijbrand · 2016-09-30T08:48:46Z

@mgedmin thank you for the comment. We processed them. Perhaps you could have another look? TIA.

mgedmin · 2016-09-30T08:58:34Z

src/zope/testing/test_renormalizing.py

-FooBarError: requires at least one argument."""
+        Traceback (most recent call last):
+        FooBarError: requires at least one argument."""
+        expected = textwrap.dedent(expected)
        self.assertEqual(expected, strip_dottedname_from_traceback(string))


This works, but what I had in mind was

def test_strip_dottedname(self): string = textwrap.dedent("""\ Traceback (most recent call last): ... foo.bar.FooBarError: requires at least one argument""") expected = textwrap.dedent("""\ Traceback (most recent call last): ... FooBarError: requires at least one argument""") self.assertEqual(strip_dottedname_from_traceback(string), expected)

Anyway, a minor matter, and while it's still a bit hard to see where a statement begins and ends, the most important thing is that now it's easy to see where a test method begins and ends. Good enough.

mgedmin

lgtm

mgedmin · 2016-09-30T08:59:26Z

CHANGES.rst

@@ -10,6 +10,13 @@ Changes

 - Cleaned up useless 2to3 conversion.

+- Introduce optionflag ``EXCEPTION_2TO3`` to normalize exception class names


This has now been renamed, yes?

Also, I think "option flag" as two words works better.

please fix this before merging! ;)

However, doctest will preprocess exception output. It gets rid of the the stack trace and the "Traceback (most recent call last)"-part. It passes only the exception message to the checker.

janwijbrand · 2016-09-30T11:05:22Z

We found out by actually using zope.testing "in the field" that being smart about detecting traceback ouput worked only in the tests of zope.testing. We found that doctest will preprocess the exception output. See commit message and comment in renormalizing.py.

mgedmin

janwijbrand added 4 commits September 29, 2016 16:37

Introduce optionflag EXCEPTION_2TO3 to normalize exception class …

62b7fa0

…names in traceback output.

be as helpful as we can in giving a hint for a failing test against t…

2f1882a

…raceback output.

tweak the hint output

8310515

allow not passing in any patterns to the renormalizing output checker.

f6cca5d

mgedmin requested changes Sep 30, 2016

View reviewed changes

janwijbrand added 5 commits September 30, 2016 09:58

use sys.version_info instead of sys.version

2690ae7

Rename EXCEPTION_2TO3 into IGNORE_EXCEPTION_MODULE_IN_PYTHON2

197cf79

use textwrap.dedent() to make long tests strings more legible.

ec65384

be more strict about how the traceback output should look like.

8d1a396

update the hint to hopefully be more helpful and explicit.

6e95071

mgedmin reviewed Sep 30, 2016

View reviewed changes

mgedmin approved these changes Sep 30, 2016

View reviewed changes

janwijbrand added 2 commits September 30, 2016 11:13

final round of cosmetics in the changelog and the textwrapping

e5f4996

We wanted to confirm more strictly we're dealing with a traceback here.

08aa3d5

However, doctest will preprocess exception output. It gets rid of the the stack trace and the "Traceback (most recent call last)"-part. It passes only the exception message to the checker.

add even more hopefully helpful explanations to the hint

5c0afda

mgedmin approved these changes Sep 30, 2016

View reviewed changes

janwijbrand merged commit 527ce11 into master Sep 30, 2016

janwijbrand deleted the doctest-2-to-3 branch September 30, 2016 12:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Doctest 2 to 3 #15

Doctest 2 to 3 #15

janwijbrand commented Sep 29, 2016

mgedmin Sep 30, 2016

janwijbrand Sep 30, 2016

mgedmin Sep 30, 2016

janwijbrand Sep 30, 2016

mgedmin Sep 30, 2016

janwijbrand Sep 30, 2016

mgedmin Sep 30, 2016

janwijbrand Sep 30, 2016

mgedmin Sep 30, 2016

janwijbrand Sep 30, 2016

mgedmin Sep 30, 2016

mgedmin Sep 30, 2016 •

edited

janwijbrand Sep 30, 2016

janwijbrand commented Sep 30, 2016

mgedmin Sep 30, 2016

mgedmin left a comment

mgedmin Sep 30, 2016

mgedmin Sep 30, 2016

janwijbrand commented Sep 30, 2016

mgedmin left a comment



		def strip_dottedname_from_traceback(string):
		# We might want to confirm more strictly were dealing with a traceback.

		@@ -10,6 +10,13 @@ Changes

		- Cleaned up useless 2to3 conversion.

		- Introduce optionflag ``EXCEPTION_2TO3`` to normalize exception class names

Doctest 2 to 3 #15

Doctest 2 to 3 #15

Conversation

janwijbrand commented Sep 29, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mgedmin Sep 30, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janwijbrand commented Sep 30, 2016

Choose a reason for hiding this comment

mgedmin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

janwijbrand commented Sep 30, 2016

mgedmin left a comment

Choose a reason for hiding this comment

mgedmin Sep 30, 2016 •

edited