typer ... utils docs --output crashes with UnicodeEncodeError on non-UTF-8 locales (e.g. Windows cp1252) when help contains emojis

[Sreekant13]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the documentation, with the integrated search.
• I already searched in Google "How to X" and didn't find any information.
• I already read and followed all the tutorials in the docs and didn't find an answer.

Example Code

# emoji_app.py
import typer

app = typer.Typer()


@app.command()
def hello(name: str):
    """Say hello 👋 to someone."""


$ typer emoji_app utils docs --output out.md
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44b' ...

Description

typer utils docs --output FILE writes the Markdown with Path.write_text(clean_docs), which uses the platform's default encoding. When the CLI's help contains non-ASCII characters - emojis are common in Typer/Rich apps - this raises UnicodeEncodeError on interpreters whose locale encoding isn't UTF-8 (e.g. cp1252 on Windows).

The fix is a one-liner - write the file as UTF-8 (which matches how the tests read it back). I have the fix plus a regression test ready and would be happy to open a PR.

Operating System

Windows

Operating System Details

Windows 11, cp1252 default locale encoding. Also reproducible on any OS with a non-UTF-8 locale, e.g. LC_ALL=C on Linux.

Project Version

0.26.8

Python Version

3.10.11

Additional Context

The crash is in typer/cli.py in the docs command: output.write_text(clean_docs) writes without an explicit encoding, so it uses the locale default (cp1252 on Windows) and fails on non-ASCII help. The fix is a one-liner - write as UTF-8 (matching how the tests read the file back). I have the fix plus a regression test ready and am happy to open a PR.

[phalberg]

Hello,
I do see this as a valid issue, a small workaround since this mostly affects windows users is to set the env variable of PYTHONUTF8 = 1 (e.g in powershell this will be $env:PYTHONUTF8 = "1"), then this works fine.
But this is quite temporary, so a fix for this should perhaps be preferred.
Ultimately up to the maintainers, but emoji's often seems to be used in these libraries.

[Sreekant13]

Thanks @phalberg, and thanks for confirming it. That PYTHONUTF8=1 tip is a solid workaround for anyone hitting this right now, since UTF-8 mode makes the file write use UTF-8 and avoids the crash.

For the actual fix I opened PR #1881: it writes the docs file as UTF-8 (one line in typer/cli.py) and adds a regression test that forces a non UTF-8 locale, so the behavior stays covered on every platform and not only Windows. Glad to take it whichever way the maintainers prefer.