Replace traceback with cgitb to improve debugging?

[jimgregory]

While I like bottle a lot, I've often had trouble debugging an application because the standard traceback does not provide much information in the stacktrace.

While t's possible to install a plugin (bottle_debugtoolbar) to help solve this problem, I've found it a bit bloated (it uses jinja templates, so it requires that module plus others) and buggy (I haven't always been able to get it to work in an existing app).

I recently came across cgitb, a module in the standard library of all current Python versions. It was originally designed to provide traceback information for cgi scripts, but can be used to debug any Python application.

The big advantage of this module is that it automatically displays:

1. the value of each argument of each function in the call stack,
2. some of the lines of code around each function call, and
3. the local variables of each function in the call stack.

This makes debugging much easier. So in a simple bottle app like this:

import bottle
app = bottle.default_app()

def error(var):
    bar = 1
    return var/bar

@app.route('/hello')
def hello():
    return error('5')

app.run(host='localhost', port=8080, debug=True)

Rather than seeing this:

Traceback (most recent call last):
File "/tmp/test_app/bottle.py", line 1000, in _handle
out = route.call(**args)
File "/tmp/test_app/bottle.py", line 2001, in wrapper
rv = callback(*a, **ka)
File "hello_app.py", line 11, in hello
return error('5')
File "hello_app.py", line 7, in error
return var/bar
TypeError: unsupported operand type(s) for /: 'str' and 'int'

You get this (the original has better indentiation):

<type 'exceptions.TypeError'>
Python 2.7.13: /usr/bin/python
Wed Dec 6 06:15:13 2017

A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.

/tmp/test_app/bottle.py in _handle(self=<bottle.Bottle object>, environ={'BROWSER': '/usr/bin/firefox', 'COLORTERM': 'truecolor', 'CONTENT_LENGTH': '', 'CONTENT_TYPE': 'text/plain', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'DESKTOP_SESSION': 'lightdm-xsession', 'DISPLAY': ':0.0', 'EDITOR': '/usr/bin/vim', 'GATEWAY_INTERFACE': 'CGI/1.1', 'GDMSESSION': 'lightdm-xsession', ...})
998 environ['bottle.route'] = route
999 environ['route.url_args'] = args
1000 out = route.call(**args)
1001 break
1002 except HTTPResponse as E:
out = None
route = <GET '/hello' >
route.call =
args = {}

/tmp/test_app/bottle.py in wrapper(*a=(), **ka={})
1999 def wrapper(*a, **ka):
2000 try:
2001 rv = callback(*a, **ka)
2002 except HTTPResponse as resp:
2003 rv = resp
rv undefined
callback =
a = ()
ka = {}

/tmp/test_app/hello_app.py in hello()
9 @app.route('/hello')
10 def hello():
11 return error('5')
12
13 app.run(host='localhost', port=8080, debug=True)
global error =

/tmp/test_app/hello_app.py in error(var='5')
5 def error(var):
6 bar = 1
7 return var/bar
8
9 @app.route('/hello')
var = '5'
bar = 1
<type 'exceptions.TypeError'>: unsupported operand type(s) for /: 'str' and 'int'
class = <type 'exceptions.TypeError'>
delattr = <method-wrapper 'delattr' of exceptions.TypeError object>
dict = {}
doc = 'Inappropriate argument type.'
format =
getattribute = <method-wrapper 'getattribute' of exceptions.TypeError object>
getitem = <method-wrapper 'getitem' of exceptions.TypeError object>
getslice = <method-wrapper 'getslice' of exceptions.TypeError object>
hash = <method-wrapper 'hash' of exceptions.TypeError object>
init = <method-wrapper 'init' of exceptions.TypeError object>
new =
reduce =
reduce_ex =
repr = <method-wrapper 'repr' of exceptions.TypeError object>
setattr = <method-wrapper 'setattr' of exceptions.TypeError object>
setstate =
sizeof =
str = <method-wrapper 'str' of exceptions.TypeError object>
subclasshook =
unicode =
args = ("unsupported operand type(s) for /: 'str' and 'int'",)
message = "unsupported operand type(s) for /: 'str' and 'int'"

The above is a description of an error in a Python program. Here is
the original traceback:

Traceback (most recent call last):
File "/tmp/test_app/bottle.py", line 1000, in _handle
out = route.call(**args)
File "/tmp/test_app/bottle.py", line 2001, in wrapper
rv = callback(*a, **ka)
File "hello_app.py", line 11, in hello
return error('5')
File "hello_app.py", line 7, in error
return var/bar
TypeError: unsupported operand type(s) for /: 'str' and 'int'

The required changes are minimal (two new lines of code, one deletion, and five changes). I can submit a pull request if there's interest.

[oz123]

can you make a PR so others can see the changes and maybe evaluate or comment on that?

[defnull]

cgitb is made for cgi scripts and won't work out of the box with bottle, because it only handles unhandled exceptions (those that would otherwise exit the interpreter loop). Bottle catches most exceptions to display the error page (and to be able to continue serving requests after an error) so the global error handler installed by cgitb would never be triggered.

The module is not very big, though. https://github.com/python/cpython/blob/3.6/Lib/cgitb.py
It would be not that hard to write plugin with a custom error handler that provides a similar level of detail for the bottle error template.

[jimgregory]

The way I've implemented it seems to work. It seems to catch and display all the errors I've tested and continues to serve requests afterwards without having to restart the server.

But I may be misinterpreting what you've said, or not thinking of all possible error cases. I've submitted a PR and would appreciate your feedback. Thanks.