Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PythonConsoleLexer does not highlight correctly custom exceptions #1548

thepabloaguilar opened this issue Sep 19, 2020 · 4 comments


Copy link

Hi, I'm having some problems with pygments because it is not highlighting my custom exceptions correctly, you can find in the following script how to reproduce the error:

from pygments import highlight
from pygments.lexers import PythonConsoleLexer
from pygments.formatters import HtmlFormatter

pycon3 = PythonConsoleLexer(python3=True, stripnl=False)
formatter = HtmlFormatter()

code = '''
>>> IOFailure(1).unwrap()
Traceback (most recent call last):

highlight(code, pycon3, formatter)  # => raises pygments.filters.ErrorToken

You'll get something like this as the output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/", line 84, in highlight
    return format(lex(code, lexer), formatter, outfile)
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/", line 63, in format
    formatter.format(tokens, realoutfile)
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/", line 95, in format
    return self.format_unencoded(tokensource, outfile)
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/formatters/", line 879, in format_unencoded
    for t, piece in source:
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/formatters/", line 710, in _wrap_div
    for tup in inner:
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/formatters/", line 728, in _wrap_pre
    for tup in inner:
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/formatters/", line 753, in _format_lines
    for ttype, value in tokensource:
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/", line 20, in _apply
    for token in filter_.filter(lexer, stream):
  File "/home/pabloaguilar/.cache/pypoetry/virtualenvs/returns-h6SXr9Wf-py3.8/lib/python3.8/site-packages/pygments/filters/", line 196, in filter
    raise self.exception(value)
pygments.filters.ErrorToken: r

As you can see in the output, the problem is the last line:

Something: returns.primitives.exceptions.UnwrapFailedError


pygments version: 2.6.1

This is not a recent problem, some people have opened issues about it:

refs dry-python/returns#568
refs sphinx-doc/sphinx#8156

Copy link

Anteru commented Sep 19, 2020

Looks like this does reproduce with 2.7.1 as well. @birkenfeld any ideas? You've looked at the referenced bugs, did we regress?

Copy link

I think I was tripped because the terminal256 formatter does not visible display error tokens (terminal does).

Easy to fix though.

Copy link

@thepabloaguilar thanks for the issue!

@Anteru I've put the changelog entry under a 2.7.2 heading; we should release this fix soon-ish but since it isn't a regression it doesn't need an immediate release, let's wait for a week or so for more potential fixes to be made.

Copy link

Thanks for the fast answer and solution! 👏🏽

Kenny2github pushed a commit to Kenny2github/pygments that referenced this issue Sep 22, 2020
@Anteru Anteru added this to the 2.7.2 milestone Oct 24, 2020
@Anteru Anteru added changelog-update Items which need to get mentioned in the changelog and removed changelog-update Items which need to get mentioned in the changelog labels Oct 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

3 participants