Repository navigation
Virtual Terminal > A2UI Pipeline using 1.26M-param model #3069
roland-id-au
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Modern terminal renderers are seriously impressive and seriously complicated. GPU glyph atlases, texture caches, custom shaders, HarfBuzz shaping, ligatures, damage tracking, grid diffing, dirty-row uploads. Alacritty, Kitty, WezTerm and Ghostty are all doing heroic work to draw what is, at the end of the day, a grid of characters really fast.
And every client still does the same thing at the end of it. Parse an escape-code stream, keep a cell grid, paint characters. Faithful, but opaque. Your phone can't reflow it, a screen reader gets a wall of box-drawing characters, and an agent has to squint at │ ▶ item │ to work out which row is selected.
So I wondered: what if instead of throwing more GPU at drawing the grid, you used a bit of AI to understand it, once, server-side? Then send the client actual UI instead of a terminal.
Write-up + demo: https://drksci.com/labs-phosphene
Code, labels, results: https://github.com/drksci/phosphene
Reddit: https://www.reddit.com/r/MachineLearning/comments/1x0gvnt/instead_of_another_gpu_terminal_renderer_i/
All reactions