Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use better-exceptions with unittest? #76

Closed
ocavue opened this issue Mar 14, 2019 · 16 comments
Closed

How to use better-exceptions with unittest? #76

ocavue opened this issue Mar 14, 2019 · 16 comments
Labels
🎁 Rewarded on Issuehunt This issue has been rewarded on Issuehunt

Comments

@ocavue
Copy link
Contributor

ocavue commented Mar 14, 2019

Issuehunt badges

I want to use better-exceptions to show error when using unittest. Here is my example:

test.py:

import better_exceptions
import unittest

better_exceptions.hook()


class MyTestCase(unittest.TestCase):
    def add(self, a, b):
        better_exceptions.hook()

        return a + b

    def test_add(self,):
        better_exceptions.hook()

        r1 = self.add(1, 2)
        r2 = self.add(2, "1")
        self.assertTrue(r1, r2)


if __name__ == "__main__":
    unittest.main()

shell:

$ export BETTER_EXCEPTIONS=1
$ echo $BETTER_EXCEPTIONS
1
$ python3 test.py
E
======================================================================
ERROR: test_add (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 17, in test_add
    r2 = self.add(2, "1")
  File "test.py", line 11, in add
    return a + b
TypeError: unsupported operand type(s) for +: 'int' and 'str'

----------------------------------------------------------------------
Ran 1 test in 0.004s

FAILED (errors=1)

Is there any way to use better-exceptions to print exception stack?


IssueHunt Summary

ocavue ocavue has been rewarded.

Backers (Total: $40.00)

Submitted pull Requests


Tips


IssueHunt has been backed by the following sponsors. Become a sponsor

@Delgan
Copy link
Collaborator

Delgan commented Mar 14, 2019

Hi @ocavue.

The .hook() call is of no use here, because the unittest module formats exceptions using traceback as you can see here: https://github.com/python/cpython/blob/master/Lib/unittest/result.py#L185

I don't know if this is possible to change this behavior.

@ocavue
Copy link
Contributor Author

ocavue commented Mar 15, 2019

Found a simple but not perfect solution:

import better_exceptions
import unittest
import sys


def wrap_test(func):
    def new_func(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as e:
            exc, value, tb = sys.exc_info()
            print(better_exceptions.format_exception(exc, value, tb))
            raise e

    return new_func


class MyTestCase(unittest.TestCase):
    def add(self, a, b):
        return a + b

    @wrap_test
    def test_add(self,):
        r1 = self.add(1, 2)
        r2 = self.add(2, "1")
        self.assertTrue(r1, r2)


if __name__ == "__main__":
    unittest.main()
$ python3 test2.py
Traceback (most recent call last):
  File "test2.py", line 9, in new_func
    return func(*args, **kwargs)
           │     │       └ {}
           │     └ (<__main__.MyTestCase testMethod=test_add>,)
           └ <function MyTestCase.test_add at 0x7fceeb6febf8>
  File "test2.py", line 25, in test_add
    r2 = self.add(2, '1')
         └ <__main__.MyTestCase testMethod=test_add>
  File "test2.py", line 20, in add
    return a + b
           │   └ '1'
           └ 2
TypeError: unsupported operand type(s) for +: 'int' and 'str'

E
======================================================================
ERROR: test_add (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test2.py", line 13, in new_func
    raise e
  File "test2.py", line 9, in new_func
    return func(*args, **kwargs)
  File "test2.py", line 25, in test_add
    r2 = self.add(2, "1")
  File "test2.py", line 20, in add
    return a + b
TypeError: unsupported operand type(s) for +: 'int' and 'str'

----------------------------------------------------------------------
Ran 1 test in 0.014s

FAILED (errors=1)

@Qix-
Copy link
Owner

Qix- commented Mar 15, 2019

@Delgan how good/bad would it be to provide an option to .hook() to override the built-in TracebackException class?

@Delgan
Copy link
Collaborator

Delgan commented Mar 17, 2019

@Qix- Honestly, I don't know, I don't have a strong opinion about this.

The sys.excepthook is intended to be replaced, this only impacts the stderr output of an application which going to crash anyway. So, there is not much risk to monkeypatch it.
Overriding a whole built-in class like TracebackException is more intrusive. However, better_exceptions also provides a patch for logging and I don't see how patching TracebackException would cause troubles. So, as it seems to be more convenient for better_exceptions end users, this would probably be considered as an improvement with unnoticeable side-effects.

TracebackException accepts others argument not implemented by better_exceptions (limit, lookup_lines, capture_locals). Overriding the class would render them no-op, but if the patch is done using an explicit option in hook(), this should not surprise user.

Also, some users may require the same improvement for unit tests run using pytest, but I don't think we are able to patch exception formatting in this case.

@ocavue In the meantime, you can also try this solution:

import unittest
import better_exceptions

def patch(self, err, test):
    return better_exceptions.format_exception(*err)

unittest.result.TestResult._exc_info_to_string = patch

@Qix-
Copy link
Owner

Qix- commented Mar 17, 2019

^ That seems like the better approach in terms of hooking, even if it's using an undocumented method. If that works @ocavue an you let us know?

@ocavue
Copy link
Contributor Author

ocavue commented Mar 18, 2019

Although it uses an undocumented method, it seems to work well on all python versions. So I think this solution is great and we should write it in README.

( I used docker-compose to test it )


v3.6_1  | E
v3.6_1  | ======================================================================
v3.6_1  | ERROR: test_add (__main__.MyTestCase)
v3.6_1  | ----------------------------------------------------------------------
v3.6_1  | Traceback (most recent call last):
v3.6_1  |   File "/usr/local/lib/python3.6/unittest/case.py", line 59, in testPartExecutor
v3.6_1  |     yield
v3.6_1  |   File "/usr/local/lib/python3.6/unittest/case.py", line 605, in run
v3.6_1  |     testMethod()
v3.6_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v3.6_1  |   File "/workdir/test.py", line 11, in test_add
v3.6_1  |     r2 = self.add(2, "1")
v3.6_1  |          └ <__main__.MyTestCase testMethod=test_add>
v3.6_1  |   File "/workdir/test.py", line 7, in add
v3.6_1  |     return a + b
v3.6_1  |            │   └ '1'
v3.6_1  |            └ 2
v3.6_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v3.6_1  |
v3.6_1  | ----------------------------------------------------------------------
v3.6_1  | Ran 1 test in 0.014s
v3.6_1  |
v3.6_1  | FAILED (errors=1)
test_better_exception_v3.6_1 exited with code 1
v3.4_1  | E
v3.4_1  | ======================================================================
v3.4_1  | ERROR: test_add (__main__.MyTestCase)
v3.4_1  | ----------------------------------------------------------------------
v3.4_1  | Traceback (most recent call last):
v3.4_1  |   File "/usr/local/lib/python3.4/unittest/case.py", line 58, in testPartExecutor
v3.4_1  |     yield
v3.4_1  |   File "/usr/local/lib/python3.4/unittest/case.py", line 580, in run
v3.4_1  |     testMethod()
v3.4_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v3.4_1  |   File "/workdir/test.py", line 11, in test_add
v3.4_1  |     r2 = self.add(2, "1")
v3.4_1  |          └ <__main__.MyTestCase testMethod=test_add>
v3.4_1  |   File "/workdir/test.py", line 7, in add
v3.4_1  |     return a + b
v3.4_1  |            │   └ '1'
v3.4_1  |            └ 2
v3.4_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v3.4_1  |
v3.4_1  | ----------------------------------------------------------------------
v3.4_1  | Ran 1 test in 0.015s
v3.4_1  |
v3.4_1  | FAILED (errors=1)
v3.7_1  | E
v3.7_1  | ======================================================================
v3.7_1  | ERROR: test_add (__main__.MyTestCase)
v3.7_1  | ----------------------------------------------------------------------
v3.7_1  | Traceback (most recent call last):
v3.7_1  |   File "/usr/local/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
v3.7_1  |     yield
v3.7_1  |   File "/usr/local/lib/python3.7/unittest/case.py", line 615, in run
v3.7_1  |     testMethod()
v3.7_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v3.7_1  |   File "/workdir/test.py", line 11, in test_add
v3.7_1  |     r2 = self.add(2, "1")
v3.7_1  |          └ <__main__.MyTestCase testMethod=test_add>
v3.7_1  |   File "/workdir/test.py", line 7, in add
v3.7_1  |     return a + b
v3.7_1  |            │   └ '1'
v3.7_1  |            └ 2
v3.7_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v3.7_1  |
v3.7_1  | ----------------------------------------------------------------------
v3.7_1  | Ran 1 test in 0.014s
v3.7_1  |
v3.7_1  | FAILED (errors=1)
v3.8_1  | E
v3.8_1  | ======================================================================
v3.8_1  | ERROR: test_add (__main__.MyTestCase)
v3.8_1  | ----------------------------------------------------------------------
v3.8_1  | Traceback (most recent call last):
v3.8_1  |   File "/usr/local/lib/python3.8/unittest/case.py", line 59, in testPartExecutor
v3.8_1  |     yield
v3.8_1  |   File "/usr/local/lib/python3.8/unittest/case.py", line 642, in run
v3.8_1  |     testMethod()
v3.8_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v3.8_1  |   File "/workdir/test.py", line 11, in test_add
v3.8_1  |     r2 = self.add(2, "1")
v3.8_1  |          └ <__main__.MyTestCase testMethod=test_add>
v3.8_1  |   File "/workdir/test.py", line 7, in add
v3.8_1  |     return a + b
v3.8_1  |            │   └ '1'
v3.8_1  |            └ 2
v3.8_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v3.8_1  |
v3.8_1  | ----------------------------------------------------------------------
v3.8_1  | Ran 1 test in 0.045s
v3.8_1  |
v3.8_1  | FAILED (errors=1)
v3.5_1  | E
v3.5_1  | ======================================================================
v3.5_1  | ERROR: test_add (__main__.MyTestCase)
v3.5_1  | ----------------------------------------------------------------------
v3.5_1  | Traceback (most recent call last):
v3.5_1  |   File "/usr/local/lib/python3.5/unittest/case.py", line 59, in testPartExecutor
v3.5_1  |     yield
v3.5_1  |   File "/usr/local/lib/python3.5/unittest/case.py", line 605, in run
v3.5_1  |     testMethod()
v3.5_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v3.5_1  |   File "/workdir/test.py", line 11, in test_add
v3.5_1  |     r2 = self.add(2, "1")
v3.5_1  |          └ <__main__.MyTestCase testMethod=test_add>
v3.5_1  |   File "/workdir/test.py", line 7, in add
v3.5_1  |     return a + b
v3.5_1  |            │   └ '1'
v3.5_1  |            └ 2
v3.5_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v3.5_1  |
v3.5_1  | ----------------------------------------------------------------------
v3.5_1  | Ran 1 test in 0.056s
v3.5_1  |
v3.5_1  | FAILED (errors=1)
v2.7_1  | E
v2.7_1  | ======================================================================
v2.7_1  | ERROR: test_add (__main__.MyTestCase)
v2.7_1  | ----------------------------------------------------------------------
v2.7_1  | Traceback (most recent call last):
v2.7_1  |   File "/usr/local/lib/python2.7/unittest/case.py", line 329, in run
v2.7_1  |     testMethod()
v2.7_1  |     └ <bound method MyTestCase.test_add of <__main__.MyTestCase testMethod=test_add>>
v2.7_1  |   File "/workdir/test.py", line 11, in test_add
v2.7_1  |     r2 = self.add(2, "1")
v2.7_1  |          └ <__main__.MyTestCase testMethod=test_add>
v2.7_1  |   File "/workdir/test.py", line 7, in add
v2.7_1  |     return a + b
v2.7_1  |            │   └ '1'
v2.7_1  |            └ 2
v2.7_1  | TypeError: unsupported operand type(s) for +: 'int' and 'str'
v2.7_1  |
v2.7_1  | ----------------------------------------------------------------------
v2.7_1  | Ran 1 test in 0.013s
v2.7_1  |
v2.7_1  | FAILED (errors=1)

@ocavue
Copy link
Contributor Author

ocavue commented Mar 18, 2019

If you guys agree with me, I can create a PR with some tests about this hooking, to make sure that future python nightly version doesn't break this behavior.

@Qix-
Copy link
Owner

Qix- commented Mar 18, 2019

@ocavue yes please. :)

@issuehunt-oss
Copy link

issuehunt-oss bot commented Jul 13, 2019

@issuehunt has funded $40.00 to this issue.


@issuehunt-oss issuehunt-oss bot added the 💵 Funded on Issuehunt This issue has been funded on Issuehunt label Jul 13, 2019
@ocavue
Copy link
Contributor Author

ocavue commented Jul 13, 2019

Interesting bot.

@ocavue
Copy link
Contributor Author

ocavue commented Jul 13, 2019

I have already posted a PR for this issue: #77

@lokesh1729
Copy link

@ocavue @Qix- does anyone need my help or this issue was almost fixed ??? I see the PR #77 not merged yet.... what are we waiting for?? it was reviewed and approved right?

@ocavue
Copy link
Contributor Author

ocavue commented Jul 17, 2019

I think the reason why #77 is not merged yet is that I didn't click the "Slove Conversation" button before. 😂

@lokesh1729
Copy link

"Solve Conversation" button ??? i never know github has such feature :P

@ocavue
Copy link
Contributor Author

ocavue commented Jul 17, 2019

I forgot what its name is. I think it has the word "solve" (not slove) in it

@issuehunt-oss
Copy link

issuehunt-oss bot commented Sep 30, 2019

@Qix- has rewarded $28.00 to @ocavue. See it on IssueHunt

  • 💰 Total deposit: $40.00
  • 🎉 Repository reward(20%): $8.00
  • 🔧 Service fee(10%): $4.00

@issuehunt-oss issuehunt-oss bot added 🎁 Rewarded on Issuehunt This issue has been rewarded on Issuehunt and removed 💵 Funded on Issuehunt This issue has been funded on Issuehunt labels Sep 30, 2019
@Qix- Qix- closed this as completed Sep 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🎁 Rewarded on Issuehunt This issue has been rewarded on Issuehunt
Projects
None yet
Development

No branches or pull requests

4 participants