Feature Request: Terminal Graphics Protocol Support (Sixel, Kitty, iTerm2)

[gwpl]

TL;DR / Abstract

Enable terminal graphics display in Claude Code by implementing support for Sixel, Kitty, and iTerm2 inline image protocols. This would allow users to view graphical output from commands (charts, diagrams, plots) directly in the terminal, enhancing data visualization and development workflows.

Problem Statement

Currently, Claude Code cannot display graphical output from terminal commands. Tools like lsix, viu, and various plotting libraries fail to render images, limiting visual feedback for:

• Data visualization (matplotlib, plotly outputs)
• Image processing results
• System diagrams and flowcharts
• Build process visualizations
• Test coverage reports with visual elements

Proposed Solution

Implement support for three major terminal graphics protocols:

1. Sixel (Six Pixels)

• Specification: DEC VT340/ANSI escape sequence based bitmap format
• Format: ESC P q [data] ESC \
• Features: 6-pixel high strips, RLE compression, palette-based colors (max 256)
• Adoption: XTerm, mlterm, foot, WezTerm, Konsole, VS Code terminal
• Pros: Widest compatibility, standardized since 1980s
• Cons: Limited color palette, no alpha channel

2. Kitty Graphics Protocol

• Format: ESC _ G [control data] ; [payload] ESC \
• Features: Full RGBA support, GPU acceleration, multiple placement modes
• Adoption: Kitty, WezTerm, Konsole
• Pros: Best performance, full color support, advanced features
• Cons: Limited terminal support

3. iTerm2 Inline Images

• Format: ESC ] 1337 ; File = [args] : [base64 data] ^G
• Features: Direct base64 encoding, supports animations, Retina displays
• Adoption: iTerm2, WezTerm, mintty, Konsole
• Pros: Simple implementation, works partially in tmux
• Cons: macOS-centric origin

Implementation Approach

1. Auto-detection: Check $TERM, $TERM_PROGRAM, and capability queries
2. Fallback chain: Kitty → iTerm2 → Sixel → Unicode blocks (▄▀)
3. Library integration: Consider using existing libraries like:
   • Rust: viuer (supports all three protocols)
   • Go: termimg
   • Python: term-image

Testing Tools

# Sixel test
convert image.jpg -geometry 800x480 sixel:-

# iTerm2 test
printf "\e]1337;File=inline=1:"; base64 -w0 image.jpg; printf "\a\n"

# Tools to verify
lsix *.png                    # Sixel image viewer
viu --features=sixel *.png    # Multi-protocol viewer
img2sixel image.png          # Direct sixel conversion

Use Cases

1. Development

   • Preview generated images/plots without leaving terminal
   • View test coverage heatmaps
   • Display architecture diagrams

2. Data Science

   • Show matplotlib/seaborn outputs inline
   • Quick data exploration visualizations
   • Model performance graphs

3. DevOps

   • System monitoring dashboards
   • Network topology diagrams
   • Container relationship visualizations

References

• Sixel Specification
• Kitty Graphics Protocol
• iTerm2 Inline Images
• Are We Sixel Yet?
• Terminal Graphics Comparison

Success Criteria

• Auto-detect terminal capabilities
• Display images from common tools (lsix, viu, img2sixel)
• Handle protocol-specific limitations gracefully
• Provide clear error messages for unsupported terminals
• Document supported terminals and protocols

This feature would significantly enhance Claude Code's capabilities for visual development workflows while maintaining backward compatibility through intelligent fallbacks.

[gwpl]

Important Clarification

This feature request primarily targets the shell command execution mode (commands run with \! prefix or via the Bash tool). Users frequently need to visualize outputs from their development workflows:

# Current state: These commands fail or show errors
\!lsix *.png
\!viu --features=sixel plot.png
\!python visualize_metrics.py  # outputs matplotlib chart
\!gnuplot performance.gp       # generates 3D plot
\!dot -Tpng graph.dot  < /dev/null |  viu -    # renders graphviz diagram
!neato -Tpng network.gv | lsix  # renders network graph
!fdp -Tpng -o /tmp/fdp.png large_graph.gv && viu /tmp/fdp.png  # force-directed graph
!echo "digraph G { A -> B -> C; }" | dot -Tpng | viu -  # inline graphviz

Immediate Benefits (Shell Commands)

• View generated plots/charts without leaving Claude Code session
• Inspect image processing results inline
• Debug visual outputs during development
• Review data visualizations from analysis scripts
• Render architecture diagrams and flowcharts with graphviz

Future Potential: Visual Reasoning Capabilities

Beyond shell command support, terminal graphics could enable Claude to participate in visual analysis alongside users:

1. Performance Analysis: Read benchmark charts together, identify bottlenecks visually
2. Data Science Collaboration: Interpret 3D plots, statistical visualizations, ML model outputs
3. Architecture Reviews: Analyze system diagrams, flowcharts, dependency graphs
4. Debugging: Examine visual test outputs, coverage heatmaps, profiling flamegraphs

This would transform Claude Code from a text-only assistant to a visual collaborator, enabling richer human-AI interaction for data-heavy and visually-oriented development tasks.

The implementation should prioritize shell command support first, with the understanding that this infrastructure could later enable multimodal reasoning capabilities.

[aaronsb]

I would really like to see this implemented! I think it would reinforce the validity of Claude Code in the terminal!

[forayconsulting]

I would love to have this! As more dev work moves to the CLI, features that eliminate context-switching are essential. Being able to instantly see visual outputs from Claude would be a huge win for keeping developers in a state of flow.

[vfarcic]

+1

[yuzutas0]

+1

[plimanowka-lm]

+1

[mmc41]

+1

[olilarkin]

+1

[Yuan-Ru-Lin]

+1

[Gyeom]

🖼️ Implementation Proposal: Terminal Image Preview Feature

I'd like to add my support for this feature and share a concrete implementation proposal.

Proposed Architecture

User attaches image
       ↓
[Terminal Capability Detection]
       ↓
┌──────────────────────────────────────┐
│ Check environment variables:         │
│ - $TERM_PROGRAM (iTerm.app, etc.)   │
│ - $KITTY_WINDOW_ID                  │
│ - $TERM (xterm-256color, etc.)      │
│ - Terminal response to XTGETTCAP    │
└──────────────────────────────────────┘
       ↓
[Select Protocol] → Kitty > iTerm2 > Sixel > Fallback
       ↓
[Display inline image with appropriate escape sequences]

Implementation Details

1. Terminal Detection (TypeScript)

interface TerminalCapabilities {
  supportsKitty: boolean;
  supportsIterm2: boolean;
  supportsSixel: boolean;
}

function detectTerminal(): TerminalCapabilities {
  const termProgram = process.env.TERM_PROGRAM;
  const kittyWindowId = process.env.KITTY_WINDOW_ID;
  
  return {
    supportsKitty: !!kittyWindowId || termProgram === 'WezTerm',
    supportsIterm2: termProgram === 'iTerm.app' || termProgram === 'WezTerm',
    supportsSixel: ['mlterm', 'xterm', 'foot', 'WezTerm'].some(
      t => process.env.TERM?.includes(t) || termProgram?.includes(t)
    )
  };
}

2. Protocol Implementation

Protocol	Escape Sequence	Max Colors	Alpha
Kitty	ESC_Gf=32,s=<w>,v=<h>;[base64]ESC\	16M	✅
iTerm2	ESC]1337;File=inline=1:[base64]^G	16M	✅
Sixel	ESCPq[sixel_data]ESC\	256	❌

3. Graceful Fallback
For unsupported terminals, keep the current [Image #N] clickable link behavior.

Use Cases

1. Screenshot debugging - See UI bugs inline
2. Chart/graph visualization - Data science workflows
3. Image diff review - Design changes in PRs
4. Diagram viewing - Architecture diagrams from tools like Mermaid

Existing Libraries

Consider using:

• Node.js: terminal-image, term-img
• Reference: WezTerm's implementation for multi-protocol support

Configuration Suggestion

{
  "terminal": {
    "imagePreview": {
      "enabled": true,
      "maxWidth": 80,
      "maxHeight": 24,
      "protocol": "auto" // or "kitty", "iterm2", "sixel", "none"
    }
  }
}

I believe this would significantly improve the developer experience, especially for:

• Debugging UI issues with screenshots
• Reviewing visual outputs from scripts
• Working with design assets

Happy to help with implementation if the team decides to move forward with this! 🚀

---

Comment generated with assistance from Claude Code

[flyq]

+1
optimizing Claude code output unrendered LaTeX formulas