Sixel Support #2496
|
Sixels are the most commonly implemented image protocol to this day. It is widely supported and often requested. Nevertheless, it is not without flaws. Let this be the canonical discussion on the merits of sixel support. |
Replies: 11 comments 14 replies
|
I am firmly against supporting sixels at the terminal level. It is an antiquated protocol, designed around a technology that no longer exists and does not hold relevance to modern technology. Ghostty should actively support alternative image protocols. In my humble opinion, the Kitty graphics protocol is the current leader and all graphics support should push that direction. In my opinion, we should actively discourage sixel support in terminals and instead encourage developers to use the kitty graphics protocol. |
|
Are there performance implications for the rest of ghostty if support were added? |
|
I'm not sure what the internal model would require for sixel support. It might not be too much overhead - it will have to detect an escape sequence, allocate the payload, decode the payload, and then decode that into pixels. Perhaps it could piggy-back on Kitty internally for the origin of an image, but for scrolls and redraw I imagine it will have some cost, because cursor placement comes into play with sixels. Perf costs are probably something that would need to be studied: it's not obvious how much of a cost it would be. |
|
Late to the game, but I didn't have access to the prerelease versions. Performance if kitty protocol is rather bad compared to sixels. I get a 3x slowdown from 0.2 to 0.6sec on average. |
|
The Kitty Image Protocol performance is a known issue. Just needs someone to come along and work on it. |
|
I've played with implementing it a bit but never got as far as getting any images displayed mainly because I ran out of time and haven't gotten back to it. It shouldn't take a whole lot more for someone to finish it off. It certainly does have limitations, but not implementing it because it's antiquated is kinda like saying that you shouldn't implement GIF support because animated PNGs are a thing. Maybe if there was some kind of intermediary that would convert Sixel graphics to Kitty graphics we could get away with not supporting it... |
|
On Fri Oct 25, 2024 at 12:54 PM CDT, Jeffrey C. Ollie wrote:
I've played with implementing it a bit but never got as far as getting any
images displayed mainly because I ran out of time and haven't gotten back to
it. It shouldn't take a whole lot more for someone to finish it off. It
certainly does have limitations, but _not_ implementing it because it's
antiquated is kinda like saying that you shouldn't implement GIF support
because animated PNGs are a thing.
Maybe if there was some kind of intermediary that would convert Sixel graphics
to Kitty graphics we could get away with not supporting it...
I think it's antiquated legacy is the reason we shouldn't implement it. It can't
convey intent in the way we want to in today's environment. Sixels transmit an
image, and if part of it is clipped, the *client* has to re-encode the image to
the clipped region. This is a huge problem not only for ssh clients, but for
clients who just don't care about the viewable region. The kitty protocol was
designed to solve these issues, and to reach back to try to support a
sixel-pixel printer and force it to work on a modern terminal brings in too many
tradeoffs. We should appreciate this tech for what it is, and move on. We have
the technology, we have the protocol, there is no reason to provide a lesser
experience in 2024.
…--
Tim
|
|
I'm not sure what the internal model would require for sixel support.
It might not be *too* much overhead - it will have to detect an escape
sequence, allocate the payload, decode the payload, and then decode
that into pixels. Perhaps it could piggy-back on Kitty internally for
the origin of an image, but for scrolls and redraw I imagine it will
have some cost, because cursor placement comes into play with sixels.
Perf costs are probably something that would need to be studied: it's
not obvious how much of a cost it would be.
Only reason I ask is because if there is no overhead and someone were
willing to maintain it, I don't see a reason not to include support, but
I'm not the guy maintaining this stuff, so my opinion is fairly moot.
…--
Tristan Partin
https://tristan.partin.io
|
|
On Fri Oct 25, 2024 at 3:27 PM CDT, Tristan Partin wrote:
Only reason I ask is because if there is no overhead and someone were
willing to maintain it, I don't see a reason not to include support, but
I'm not the guy maintaining this stuff, so my opinion is fairly moot.
Yeah, I understand that sentiment. The issue I see is that sixel is
underspecified itself which makes implementation a moving target.
[Here](hpjansson/chafa#192) is a lengthy discussion on
at least one of the issues.
…--
Tim
|
|
On Fri Oct 25, 2024 at 3:36 PM CDT, Tim Culverhouse wrote:
On Fri Oct 25, 2024 at 3:27 PM CDT, Tristan Partin wrote:
> Only reason I ask is because if there is no overhead and someone were
> willing to maintain it, I don't see a reason not to include support, but
> I'm not the guy maintaining this stuff, so my opinion is fairly moot.
Yeah, I understand that sentiment. The issue I see is that sixel is
underspecified itself which makes implementation a moving target.
[Here](hpjansson/chafa#192) is a lengthy discussion on
at least one of the issues.
Great point. Then I think not supporting it makes sense if it will cause
headaches for maintainers.
…--
Tristan Partin
https://tristan.partin.io
|
|
My stance for now is that Ghostty will not support sixels. This is a departure from our general pattern of supporting all manner of arcana from terminals past. It is also a departure from our pattern of supporting things xterm has with few questions. But those patterns are just that: patterns, they aren't law. In this case, I'm making an informed decision based on my own research plus feedback here that sixels aren't worth it. They have numerous unspecified edge cases (some noted in this thread), the libsixel library is poorly written and poorly suited for drop-in adoption, the performance impacts are unclear but unlikely zero, and there is a feature rich future protocol that Ghostty already supports (the Kitty Graphics Protocol). This decision isn't final (as circumstances change, so do my opinions indefinitely), but I think it's definitive enough to answer this topic. I welcome any further discussion and feedback about it. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
I'd like to add to this discussion the fact that tmux supports sixels but not the other image protocols, with AFAIK no plans to do so either. So for tmux users, Ghostty will effectively lack image support until/unless it gains sixel support. And while alternatives to tmux exist, for various reasons many continue to require it or prefer it to any of those existing alternatives. |
|
Thanks! I'll read up on that and see if I can configure things to get it working, as it doesn't "out of the box" with the tools I'm using. |
|
I can confirm that Ghostty + tmux works perfectly with Kitty graphics :) Try timg 
|
|
how can I preview images in ranger using kitty protocol? |
This is not the place to ask make a new discussion in the future. Ranger recently merged support for all terminals with kip support but I assume thats not released into a version yet for now you have to open ranger with |
|
just use yazi |
|
I think terminal multiplexers are still a valid argument for supporting Sixel. The Kitty graphics protocol is feature-rich, well-defined, but also quite complex, this complexity affects its adoption speed. Currently, the two popular terminal multiplexers, tmux and Zellij, do not support it, but they both support Sixel:
|
|
You should check the actual state of sixel support in these multiplexers. Yes, they do advertise sixel support but how good is it in various scenarios? (I'm saying this not out of speculation but personal experience) Then, take that and compare it to what you get via kitty graphics' unicode placeholders. Also, consider this: In a remote tmux session, scroll the terminal with an image on screen and compare the amount of bytes transferred over the network and the redraw performance. To be honest, instead of pushing for sixel support here, anyone should be pushing for passthrough support in Zellij (and any other multiplexer that lacks it). For what's worth, nothing stops passthrough from working in arbitrarily nested sessions. You only need to nest the passthrough sequence accordingly. As for detecting nested sessions, |
|
My intention is not to push anything; I was just passing by to add some supplements to the previous discussion. If these contributions are unhelpful and create noise, I apologize. I don't think it's very meaningful to compare Sixel and Kgp here. This discussion is about “adding extra Sixel support”, not “using Sixel to replace the existing Kgp”. Adding Sixel means that both users and developers have a new option available. If one must compare, in my view: Kgp > Sixel > Kgp via passthrough.
This is the crux of the problem. The existing Unicode placeholders need to know the tmux nesting level and nest the passthrough sequence accordingly, which requires querying According to the tmux maintainer, tmux/tmux#4386 (comment):
And once an application starts using passthrough to receive responses, it's like opening Pandora's box: you have to handle a series of issues it brings along:
|
Oh, not at all. I apologise for my tone earlier. 🙏 To be clear, my major concern is whether the maintenance burden is worth what we get out of it. Thank you so much for the corrections and clarifications about using passthrough. |
|
No worries! I understand the concerns about maintenance burden — I run some open source projects myself, and sometimes I have to make trade-offs and drop certain features to avoid making maintenance an endless headache. That said, if there ever comes a day when Sixel support is reconsidered and my input helps in the decision-making, I'd be truly honored. 😉 |
This comment was marked as disruptive content.
This comment was marked as disruptive content.
My stance for now is that Ghostty will not support sixels.
This is a departure from our general pattern of supporting all manner of arcana from terminals past. It is also a departure from our pattern of supporting things xterm has with few questions. But those patterns are just that: patterns, they aren't law.
In this case, I'm making an informed decision based on my own research plus feedback here that sixels aren't worth it. They have numerous unspecified edge cases (some noted in this thread), the libsixel library is poorly written and poorly suited for drop-in adoption, the performance impacts are unclear but unlikely zero, and there is a feature rich future protocol that Ghostty alre…