Skip to content

feat(tui): add ambient terminal pets#21206

Merged
fcoury-oai merged 42 commits into
mainfrom
tui-pets
May 12, 2026
Merged

feat(tui): add ambient terminal pets#21206
fcoury-oai merged 42 commits into
mainfrom
tui-pets

Conversation

@fcoury-oai
Copy link
Copy Markdown
Contributor

@fcoury-oai fcoury-oai commented May 5, 2026

Why

The Codex App has animated pets, but the TUI had no equivalent ambient companion surface. This brings that experience into terminal Codex while keeping the main chat flow usable: the pet should feel present, but it cannot cover transcript text, composer input, approvals, or picker content.

The feature also needs to be terminal-aware. Different terminals support different image protocols, tmux can interfere with image rendering, and some users will want pets disabled entirely or anchored differently depending on their layout.

CleanShot 2026-05-05 at 12 41 45@2x

macOS - Ghostty, iTerm2 and WezTerm with Custom Pet

![Uploading CleanShot 2026-05-10 at 20.28.30.png…]()

Windows Terminal

CleanShot 2026-05-05 at 12 39 02@2x

Linux - WezTerm and Ghostty

What Changed

  • Add a TUI ambient pet renderer in codex-rs/tui/src/pets/.
  • Port the app-style pet animation states so the sprite changes with task status, waiting-for-input states, review/ready states, and failures.
  • Add /pets selection UI with a preview pane, loading state, built-in pet choices, and a first-row Disable terminal pets option.
  • Download built-in pet spritesheets on demand from the same public CDN path already used by Android, under https://persistent.oaistatic.com/codex/pets/v1/..., and cache them locally under ~/.codex/cache/tui-pets/.
  • Keep custom pets local.
  • Add config support for pet selection, disabling pets, and choosing whether the pet follows the composer bottom or anchors to the terminal bottom.
  • Reserve layout space around the pet so transcript wrapping, live responses, and composer input do not render underneath the sprite.
  • Gate image rendering by terminal capability, disable image pets under tmux, and support both Kitty Graphics and SIXEL terminals.
  • Add redraw cleanup for terminal image artifacts, including sixel cell clearing.

Current Scope

  • This is an initial TUI version of ambient pets, not full App parity.
  • It focuses on ambient sprite rendering, /pets selection, custom pets, terminal capability gating, and on-demand CDN-backed built-in assets.
  • The ambient text overlay is currently disabled, so the TUI renders the pet sprite without extra status text beside it.

How to Test

  1. Start Codex TUI in a terminal with image support.
  2. Run /pets.
  3. Confirm the picker shows built-in pets plus custom pets, and the first item is Disable terminal pets.
  4. On a fresh ~/.codex/cache/tui-pets/, move onto a built-in pet and confirm the first preview downloads the spritesheet from the shared Codex pets CDN and renders successfully.
  5. Move through the pet list and confirm subsequent built-in previews use the local cache.
  6. Select a pet, then send and receive messages. Confirm transcript and composer text wrap before the pet instead of rendering underneath the sprite.
  7. Change the pet anchor setting and confirm the pet can either follow the composer bottom or sit at the terminal bottom.
  8. Return to /pets, choose Disable terminal pets, and confirm the sprite disappears cleanly.

Targeted tests:

  • cargo test -p codex-tui ambient_pet_
  • cargo test -p codex-tui resize_reflow_wraps_transcript_early_when_pet_is_enabled
  • cargo insta pending-snapshots

@fcoury-oai fcoury-oai requested a review from a team as a code owner May 5, 2026 15:24
@fcoury-oai fcoury-oai force-pushed the tui-pets branch 6 times, most recently from 5259c6f to 15bfa0f Compare May 10, 2026 20:38
fcoury-oai added a commit that referenced this pull request May 10, 2026
fcoury-oai added a commit that referenced this pull request May 10, 2026
fcoury-oai added a commit that referenced this pull request May 10, 2026
Comment thread codex-rs/tui/src/pets/catalog.rs Outdated
@fcoury-oai
Copy link
Copy Markdown
Contributor Author

@etraut-openai I have internalized the Sixel processing along with the rest of your feedback. Ready for another pass.

Copy link
Copy Markdown
Collaborator

@etraut-openai etraut-openai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really large PR, which makes it difficult to review in its entirety.

I did skim through the code, and I found a few issues (some with the help of codex).

I did a bunch of manual testing in various terminals — both those that support the graphics escapes and those that don't (all on macOS). It seems to work well.

I also verified that the total binary size increase. It's about 0.92MiB or a 0.5% increase, which is somewhat high for a single TUI feature, but I think it's within reason.

Comment thread codex-rs/tui/src/pets/mod.rs
Comment thread codex-rs/tui/src/pets/mod.rs Outdated
Comment thread codex-rs/tui/src/chatwidget.rs Outdated
Comment thread codex-rs/tui/src/app/event_dispatch.rs
Comment thread codex-rs/tui/src/chatwidget/tests/status_and_layout.rs
@fcoury-oai
Copy link
Copy Markdown
Contributor Author

@etraut-openai thanks a lot for the review. I addressed them all, please do another pass when you can.

Copy link
Copy Markdown
Collaborator

@etraut-openai etraut-openai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good.

I built the latest and did a quick smoke test.

I think there's room for some additional simplification, so consider running "simplify" pass on it before merging. See slack DM for details.

@fcoury-oai fcoury-oai enabled auto-merge (squash) May 11, 2026 23:35
@fcoury-oai fcoury-oai disabled auto-merge May 11, 2026 23:38
@fcoury-oai fcoury-oai merged commit 95b332c into main May 12, 2026
38 of 39 checks passed
@fcoury-oai fcoury-oai deleted the tui-pets branch May 12, 2026 13:43
@github-actions github-actions Bot locked and limited conversation to collaborators May 12, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants