[BUG] Unable to handle UnicodeEncodeError in RichHandler #3600
Description
- I've checked docs and closed issues for possible solutions.
- I can't find my issue in the FAQ.
Describe the bug
When logging invalid utf8 with RichHandler, the logging breaks completely. Even when applying the provided fix. MWE:
import logging
from rich.logging import RichHandler
logger = logging.getLogger()
logger.setLevel("INFO")
# console_handler = logging.StreamHandler() # This works
console_handler = RichHandler() # This throws an exception
console_handler.setLevel("INFO")
logger.handlers = [console_handler]
invalid_output = "\udcf1"
logger.info(invalid_output)This is the output:
$ PYTHONIOENCODING=utf-8 python testrich.py
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/rich/logging.py", line 170, in emit
self.console.print(log_renderable)
File "/usr/lib/python3/dist-packages/rich/console.py", line 1673, in print
with self:
File "/usr/lib/python3/dist-packages/rich/console.py", line 865, in __exit__
self._exit_buffer()
File "/usr/lib/python3/dist-packages/rich/console.py", line 823, in _exit_buffer
self._check_buffer()
File "/usr/lib/python3/dist-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf1' in position 60: surrogates not allowed
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Call stack:
File "/home/adrian/git/New-Solvency/testrich.py", line 15, in <module>
logger.info(invalid_output)
File "/usr/lib/python3.12/logging/__init__.py", line 1539, in info
self._log(INFO, msg, args, **kwargs)
File "/usr/lib/python3.12/logging/__init__.py", line 1684, in _log
self.handle(record)
File "/usr/lib/python3.12/logging/__init__.py", line 1700, in handle
self.callHandlers(record)
File "/usr/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
hdlr.handle(record)
File "/usr/lib/python3.12/logging/__init__.py", line 1028, in handle
self.emit(record)
File "/usr/lib/python3/dist-packages/rich/logging.py", line 172, in emit
self.handleError(record)
Message: '\udcf1'
It looks like a duplicate of #212, but I'm opening a separate bug because as you can see, the error message is suggesting to set PYTHONIOENCODING=utf-8, but it doesn't work. I don't find this to be an acceptable solution anyway, because I cannot request my users to set this variable. There should be some graceful failure mode.
Platform
Click to expand
I'm using Debian sid. My terminal is Konsole. My shell is zsh.
$ .venv/bin/python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface. │
│ │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=88 ColorSystem.TRUECOLOR> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ color_system = 'truecolor' │
│ encoding = 'utf-8' │
│ file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│ height = 26 │
│ is_alt_screen = False │
│ is_dumb_terminal = False │
│ is_interactive = True │
│ is_jupyter = False │
│ is_terminal = True │
│ legacy_windows = False │
│ no_color = False │
│ options = ConsoleOptions( │
│ size=ConsoleDimensions(width=88, height=26), │
│ legacy_windows=False, │
│ min_width=1, │
│ max_width=88, │
│ is_terminal=True, │
│ encoding='utf-8', │
│ max_height=26, │
│ justify=None, │
│ overflow=None, │
│ no_wrap=False, │
│ highlight=None, │
│ markup=None, │
│ height=None │
│ ) │
│ quiet = False │
│ record = False │
│ safe_box = True │
│ size = ConsoleDimensions(width=88, height=26) │
│ soft_wrap = False │
│ stderr = False │
│ style = None │
│ tab_size = 8 │
│ width = 88 │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available. │
│ │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│ │
│ truecolor = False │
│ vt = False │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ { │
│ 'TERM': 'screen-256color', │
│ 'COLORTERM': 'truecolor', │
│ 'CLICOLOR': None, │
│ 'NO_COLOR': None, │
│ 'TERM_PROGRAM': 'tmux', │
│ 'COLUMNS': None, │
│ 'LINES': None, │
│ 'JUPYTER_COLUMNS': None, │
│ 'JUPYTER_LINES': None, │
│ 'JPY_PARENT_PID': None, │
│ 'VSCODE_VERBOSE_LOGGING': None │
│ } │
╰────────────────────────────────────╯
platform="Linux"
$ uv pip freeze | grep rich
rich==13.9.4