This is fun and i like it! #49
Comments
|
Good question! I'd love to have an answer to that on the lsix page. I mostly use For the Fastest Sixel Shooter in the West, I've seen some impressive sixel demos from @dankamongmen, so you may want to check out notcurses. @j4james: any thoughts on a good sixel benchmark? It would be nice to have a good one for whatever test suite comes out of the vt340test. I'm not sure if this will be helpful, but I wrote a gif viewer that measured frames per second by sending an inquiry to the terminal and presuming the image had finished rendering once it got a response. |
|
While I appreciate the kind words from @hackerb9 , Notcurses's sixel implementation as of 2.3.x leaves some room for improvement. I'd say the current state-of-the-art is libsixel, which is (on many images) faster and more accurate than the homegrown Notcurses solution (with that said, Notcurses can handle some images that cause libsixel to fail). I don't expect this to remain the case for very long, but it's true at the moment. With regards to the Kitty graphics protocol, Notcurses is probably the most advanced. I have some timing data here for Note that there are some design tradeoffs that can affect raw benchmarks. WIth Kitty, you can either sideload a file or transmit its contents directly; the former is obviously faster, but only works on the local machine. You can use or not use compression; it slows down your image encoding, but can speed up use over a network (though probably not if you already use compression with OpenSSH). With Sixel, you can share palettes among multiple images, which cuts overhead but is obviously only a win when there are multiple images available. There's then the issue of startup time -- a program with high startup time can see it amortized over multiple loads, but will look bad if invoked once per image. Finally, how much parallelism is used? High parallelism might look great when run by itself, but not look so good when the machine is otherwise loaded. Etc, etc, etc. With smaller images, it ought be entirely possible to drive hundreds of frames per second. |
|
it is interesting, for instance, to run |
|
oh if you were asking about fastest display as opposed to encoding, Sixel is fastest on WezTerm and foot in my experience. Kitty is fast on both Kitty and WezTerm, the only two implementations of which i am aware. |
in addition to time performance, it would be awesome to compare quantized image with original. i've got a bug on this here: dankamongmen/notcurses#1724 |
|
I haven't really looked, but the only sixel benchmark I've come across was https://github.com/jerch/sixel-bench, which just measures the throughput of a video sequence that's been pre-encoded as sixel. It's not really a good way to compare implementations though, because it doesn't account for the rendering frame rate. For example, if your renderer is the bottleneck, you can improve your throughput dramatically just by dropping 90% of the frames, but that's not what most people would consider an improvement. That said, I do think it could still be useful for a terminal developer evaluating improvements in their own implementation, as long as they understand what they're actually measuring. |
@j4james nice touch! |
|
So, I just tried my ancient sixvid gif viewer on the sixel capable terminals that are packaged with the current Debian GNU/Linux (11, bullseye) and got some results that were not what I expected.
|
|
Here's my sixvid gif viewer if you'd like to see relative speeds of sixel implementations on your own computers: https://github.com/hackerb9/sixvid Usage: (Hit the |
interesting. here's
notcurses 2.3.17. were you running |
|
if we look beyond sixel:
|
|
i put these stats up at https://nick-black.com/dankwiki/index.php?title=Notcurses#Pixel_blitters. @kovidgoyal, i know you like this kind of thing =] |
|
Yeah thanks, and am happy to see no surprises there, and this is without using side band transmission even, I think? |
|
I couldn't get sixvid to work - the screen just went black. I don't know whether it's dependent on something I don't have installed. I didn't spend much time trying to figure it out, but will give it another go tomorrow. In the meantime, though, I have run sixel-bench on some of the terminals in my collection. As I mentioned above, some will just drop frames (sometimes all frames), so when that's obviously the case I've just discounted them from the running (that includes Rxvt, St, and Yaft). I've also not included any Windows terminals, because I wanted to limit the competition to the same VM. All of the ones below at least gave the appearance of rendering all the frames, so I think there's a reasonable chance they're competing fairly, but I wouldn't read too much into these results. Ordered from fastest to slowest:
I don't understand why WezTerm was so slow given the earlier praise it got from @dankamongmen - I did download the latest nightly to see if that made any difference, but no such luck. I don't know whether perhaps my use of a VM might effect some terminals more than others. At any rate, VTE looks to me to be the winner by a long way. It's possible it gets that speed from not rendering all the frames, but it wasn't obviously doing anything like that. It just looked fast and smooth. |
A good question and one I am gratified to be able to answer promptly: I do not know. It runs at the same speed when I unset the DISPLAY, if that means anything. |
Huh. I wonder if ImageMagick is calling |
|
j4james: my praise for WezTerm is for its kitty implementation. I recall it being unremarkable with sixel, though not as poor as your results would suggest. if you built from source, did you use "cargo build --release"? unoptimized rust is very very slow. that VTE number shocks me. I didn't even think VTE implemented sixel? |
so foot, AFAIK, is a wayland-only terminal. the others are mostly X, though a few can do both. I'm not sure how you would run foot in Xorg without running Xwayland, interesting. |
correct, Notcurses eschews zee sideband |
|
with that said, my sixel vs kitty numbers oughtn't be taken out of the Notcurses context. my sixel quantization algorithm, as I've mentioned, is not where I'd like it to be. with that said, the kitty 0.19.3 numbers are very compare to the best sixel numbers, but the kitty 0.23.1 runs crush them. this is due to both improvements in kitty's internals, improvements in the kitty protocol, and Notcurses taking advantage of those improvements. with that said, I'm a big believer in the superiority of the kitty protocol in just about every sense, even (perhaps counter-intuitively) implementation complexity (no need to deal with heights that aren't multiples of 6, no need to avoid the bottom row, no need to quantize). |
Nope, I just downloaded the nightly deb package from github and installed that. Many of the others were built from source, though, so that's definitely worth bearing in mind. A different compiler could easily make a difference to the performance. As I said, don't read too much into my results.
Like XTerm, you've got to build it yourself with the appropriate option enabled. I don't think you'll find it in a released package anywhere.
Technically you shouldn't need to avoid the bottom row with sixel either - that's just a bug in most implementations. |
|
@j4james I've put in some sanity checks so hopefully you'll get an error message now instead of a blank screen. |
huh. that number seems very suspect, as i've found the total time to generally be dominated by transfer, not by actual rendering. i'm likewise surprised that mlterm could possibly score so high, as it's consistently been the slowest in my testing. very strange. |
oh absolutely, i'd love for that to be the case. i have a good hundred lines of code devoted to dealing with this annoyance. kitty meanwhile has |
meanwhile the linux framebuffer console keeps drawing kinda-but-not-completely distinct from text, so i have do all my image scrolling manually there =] |
|
@dankamongmen wrote:
Interesting. I added a |
Since, IIRC, times are dominated by encoding/transmission/decoding using the sideband should yield substantial improvements. Most well designed terminal emulators should not have rendering as a bottleneck. |
|
@dankamongmen writes:
You'll find no argument. In fact, I think they may not even be comparable. It's not just that sixel is a protocol from 30 years ago. The kitty graphics protocol became a completely different type of thing once it added the notion of tagged images. Sixel splats bitmaps on the screen and forgets about them. A terminal that supports the kitty protocol must treat images as first class citizens, on the same level as text. That's a huge paradigm shift. While I have some questions — how do sixel and kitty graphics interact? why can't text erase graphics? how do graphics interact with VT220 scrolling windows? do the images disappear as one would expect when switching text pages? does switching back making them reappear? why does loop=2 mean loop once, and loop=1 mean ∞? — I think kitty is the most probable future. @kovidgoyal has done a yeoman's job with the kitty graphics protocol and I look forward to it becoming standardized. In the meantime, though, I'm having fun with sixels. ImageMagick understands them so I can do quick tests from the command line as I manipulate images or write tiny shell scripts like |
|
On Sat, Aug 28, 2021 at 10:59:34PM -0700, hackerb9 wrote:
While I have some questions — _how do sixel and kitty graphics interact?
They dont :)
why can't text erase graphics?
Because graphics can be over or under text and really have an existence
independent of text.
how do graphics interact with VT220 scrolling windows?
You mean margins? https://sw.kovidgoyal.net/kitty/graphics-protocol/#interaction-with-other-terminal-actions
do the images disappear as one would expect when switching text pages?
If you mean the alternate screen, yes the main and alternate screen
maintain their own independent image lists.
does switching back making them reappear?
to the main screen yes.
why does loop=2 mean loop once, and loop=1 mean ∞?_
Because we are only allowed positive numbers and 0 means *not
specified*. So 1 means infinite and numbers higher than that mean
a finite number (n - 1).
|
This time I tried my |
Similar, but not quite. I meant Page Memory, which is how sixel does double-buffering. You can choose which page to write on and which page to display and they don't have to be the same. |
|
There's been some performance improvements to the sixel decoder in foot since 1.6.4. Still, sixvid with nyancat is much slower than I'd expect. This is something I'm going to want to look into. (all benchmarks run on lousy laptop)
|
|
@dnkl wrote:
Uh. What happened to your XTerm? 17 frames per second on nyantocat? I just pulled out an old 32-bit Pentium M laptop (circa 2005) to test and I get better FPS than that. (Not by much, mind you. But, still...) Surely your laptop isn't more than fifteen years old, right? |
Good question... turned out to be this: With the default |
|
And with |
|
I took a quick look at the nyancat issue, but not really sure what's going on; none of the processes are using nowhere near 100% CPU. My best guess atm is that sixvid is stalling on a full PTY pipe, since foot does not consume any PTY data while rendering. However, I would expect much higher CPU usage from foot if that was the case, as well as long rendering times. But most frames are rendered in less than 1ms. I'll have to sit down one day and dig deeper into this. @hackerb9 thanks for a, what it looks like, very interesting benchmark :D |
|
On Sun, Aug 29, 2021 at 01:48:35AM -0700, hackerb9 wrote:
> If you mean the alternate screen, yes the main and alternate screen maintain their own independent image lists.
Similar, but not quite. I meant [Page Memory](https://archive.org/details/bitsavers_decterminaT330VT340TextProgrammingMar87_54100022/page/n116/mode/1up?), which is how sixel does double-buffering. You can choose which page to write on and which page to display and they don't have to be the same.
kitty doesnt implement paged memory, it has a continuous scrollback
buffer instead. Images are preserved in the scrollback, up to some limit
which the spec leaves upto implementations but recommends that it be at
least several screenful of images. kitty stores image data encrypted
with a one time key on disk, so its limits are fairly generous, dont
recall what they are off the top of my head.
The protocol doesnt talk about paged memory at all. If some terminal
emulator wants to implement it and also the graphics protocol, I am happy
to add to it.
The protocol includes ways to query the terminal emulator to check if an
image is resident or not. And when the terminal emulators memory limits
are being reached it is supposed to evict images that are the "oldest"
aka furthest back in the scrollback.
|
Thanks - that helped a lot. It turns out I just didn't have ffmpeg installed on my test VM. I'm not going to list specific numbers this time, because there was quite a lot of fluctuation in the frame rates, but the results were in the same ballpark as the sixel-bench test. VTE is still at the top, and WezTerm and DomTerm are still clearly at the bottom. The midfield were fairly close, but if I had to order them, I'd consider Alacritty and Contour the leaders of that group now, with Xterm and MLTerm bringing up the rear (which seems more in line with the results @dankamongmen was seeing). That said, something that came up in the sixvid test that wasn't apparent with sixel-bench, is that Alacritty eats through memory like there's no tomorrow. After running sixvid for a minute or so it had chewed up several gigs of memory and died. Sixvid also enabled me to get what I think was a fairer test of Rxvt, St, and Yaft, since they were now actually displaying all the frames (or at least appeared to be). The first two didn't do very well though - around the same speed as WezTerm - and Rxvt had the same memory-eating issue as Alacritty. However, Yaft's performance was fantastic - it seems about 50% faster than VTE even. It is a framebuffer terminal, so maybe that's a factor, and it's possible it's just not showing all the frames, but from a user point of view it looked great. |
Yeah, I saw that from some of the terminal emulators. I had expected that to only happen from videos with differing levels of compressibility, but not nyantocat. Anyhow, I've added a final FPS reading when you quit the program which will give you an overall frames per second, starting from the point where the decoding and sixelizing finished.
Yipes!
Wow. I had benchmarked yaft as plenty fast, but middling on my machine. Are you using any sort of special framebuffer in the VM that might affect the performance measurement? Yaft lacks a scroll back buffer, right? |
That's brilliant. Thanks.
Not that I'm aware of. But maybe it's just that the GUI-based terminals are at a disadvantage running in the VM because they're not getting the video acceleration they would usually get. The results for WezTerm in particular seem hard to believe. Also I know I can't run foot at all, because it depends on Wayland, which doesn't work in my VM. And perhaps if I did have Wayland that would give a performance boost to some terminals that are otherwise limited by X11.
Yeah I think so. There's no concept of a scrollbar, and mouse wheel scrolling doesn't seem to do anything. |
I've fixed the FPS calculation to take much less time to converge. Previously, you had to wait quite a while before it gave the correct answer.
What is your VM set up? Perhaps I or someone else can replicate your results. |
The VM is Hyper-V, running on Windows 10.0.18363.1500. It's got 2 virtual processor, and it's allocated 3GB of memory (but with the dynamic memory option enabled, so I think it can use to more than that). The guest OS is Ubuntu 20.04, but it wasn't installed as that initially (it's been upgraded a couple of time). I probably should try doing a clean install from the VM "quick create" setup, but I'm not convinced that'll make any difference. |
I can't replicate Hyper-V as I don't have Microsoft Windows. Do you have an old junker computer or laptop you could install Ubuntu onto? I think that'd make a bigger difference than trying to reinstall within Hyper-V. Alternately, you could use a Live USB stick to try Ubuntu on the bare hardware without affecting your Microsoft Hyper-V installation. |
Neither of those options are feasible for me, and I'm not that enthusiastic about performance testing anyway. I was just curious to get a general idea of the speed of other terminals to compare with my own crappy implementation. Personally I care more about the correctness of the Sixel emulation, and if I'm close to the middle of the range in performance, I'd consider that a win. |
More than fair enough. I look forward to working with you more on correctness of sixel implementations. |
What's the fastest sixel viewer?
The text was updated successfully, but these errors were encountered: