Lowered performance on macOS with 4K display and "Scaled" resolution

[jhrmnn]

The way macOS handles scaling UI elements on high-resolution displays is to render everything in double the size and scale down. For instance, to achieve a reasonable UI element size on a 4K 28" display, many people would run the display at virtual 2560x1440. But this means that macOS renders at ~5K and scales down. This can be observed on Kitty. Scrolling down in Vim (just holding "j") opened across the full 4K display scaled down to 1440p gives me FPS of ~25 (MacBook Pro, early 2015, Intel Iris 6100). Doing the same at the full non-scaled 4K resolution gives me much more reasonable FPS of ~50.

Importantly, this is all strictly tied to the size of the area rendered by Metal. With the Kitty window occupying only half of the display (at 1440p), I get almost the same FPS as with the full-size window at the non-scaled 4K resolution.

Also, not everything in macOS is rendered in this scaled-down fashion. For instance, pictures or video are always rendered at the native display resolution, pixel-to-pixel, regardless of the scaling of UI elements.

This leads me to the question whether Kitty couldn't optionally render the terminal window in the same way that pictures and video are rendered, ignoring the resolution scaling, and thus avoiding the performance-hitting scaling-up. This would make the font size inconsistent with the UI-rendered font sizes, and one would need to select a larger font size to compensate, but that's fine I guess.

What I don't know is if Metal can be used to render fonts in this non-scaled fashion, or whether that is somehow hardcoded in the system.

Btw, thanks for making Kitty, it is an amazing piece of software.

[kovidgoyal]

I'm not a macOS user/developer so I'm not sure about what follows next, I could be wrong. As far as I know, kitty does not request the OS to do scaling. It maybe there is some special API it can use to tell the OS to not scale it, but I dont know what it is, and I dont know what effect it would have when, say moving kitty windows between monitors with different DPI, for example. Since the primary use of scaling, as far as I know is for different DPI display devices. I'm somewhat surprised to hear you need to turn on scaling at all. Most modern applications handle high DPI displays automatically, adjusting UI elements anf font sizes by the DPI. kitty does this as well.

That said, a quick google, found: https://apple.stackexchange.com/questions/55486/how-do-i-disable-scaling-for-specific-apps-on-retina-macbook-pro

maybe that will do the trick for you. If you or anyone else can come up with the API needed to do that from within kitty, I will be happy to implement a macos specific option to turn it on in kitty. If so, just post here.However, I think the proper solution for this, is to just nbot use scaling. It should be unneccessary unless you use lots of legacy applications that are not DPI aware.

[jhrmnn]

I looked a bit more into this. I was wrong in assuming that some UI elements can be selectively rendered at the native resolution. What's happening is the following.

As far as I know, kitty does not request the OS to do scaling.

It doesn't, macOS does it for the whole desktop canvas automatically.

Most modern applications handle high DPI displays automatically, adjusting UI elements anf font sizes by the DPI.

Not in macOS, afaik. macOS cannot really scale up the UI elements, it can only render everything in double the DPI (for Retina). So to simulate any resolution in between ordinary (1x) and Retina (2x), it renders everything in Retina DPI on an even large canvas than when run at native resolution, and scales down the resulting raster. In the picture below, the left column is a physical screen of a fixed size, in the middle the virtual display set by the user, and in the right column the actual rendered canvas. First row is a low-resolution display, the second is 4K run in Retina mode, third 4K in native resolution, and last row is my case, 4K at scaled resolution. You see that my case requires the largest render of all.

Given all this, I don't think anything can be done. It's simply a limitation of how macOS handles scaling UI elements. The only solution is to get a faster GPU or a 5K monitor of the same physical size, whose Retina mode corresponds to my desired user resolution.

[i512]

Interestingly, kitty was really smooth for me on 0.11.3 with the same setup (4k + scaling), while any other terminal didn't even come close to being usable.

I notice that there is a serious performance regression, introduced in 0.12.0.
Did you expect a performance regression from new features in that version or is this a bug?

[jhrmnn]

I can confirm a regression in FPS on 4K scaled to 1440p from 0.11.3 to 0.12.0 by about 25%. All newer version up to 0.13.1 have the same performance as 0.12.0.

Given my understanding of the problem, I think this implies a general regression on macOS, regardless of the screen resolution. It's most likely just not noticeable on lower or non-scaled resolutions.

[jhrmnn]

Unless @kovidgoyal has a guess where this could come from, I'll try to bisect it over Christmas.

[kovidgoyal]

No idea, sorry. How are you measuring FPS? There have been a lot of changes since 0.11.3

[jhrmnn]

I have a little monitoring tool in macOS (iStat Menus) which shows FPS in one of its widgets. I scroll in Vim by pressing “j” (which pushes Kitty to 100% CPU on the scaled 4K) and watch the FPS.

…

On 17. Dec 2018, at 22:19, Kovid Goyal ***@***.***> wrote: No idea, sorry. How are you measuring FPS? There have been a lot of changes since 0.11.3 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[Luflosi]

@kovidgoyal How difficult would it be to implement a frame time display that measures how long it takes to render each frame, similar to Alacritty?

[kovidgoyal]

not too difficult, but not something I care much about. For me performance is more about CPU usage during everyday tasks, than FPS. That said, it will certainly be useful to have an inbuilt FPS meter

[Luflosi]

I said frame time and not FPS for a reason. FPS is usually limited by the monitor and by a config option in kitty itself. Frame times show how long it took to render a frame, which has nothing to do with FPS if the FPS is artificially limited like it is most of the time. The frame time would also correlate with the CPU/GPU usage. If the frame time is higher, the CPU/GPU usage is also higher. It might also be useful for quick benchmarks.

[Luflosi]

If course FPS is just the inverse of the frame time if there is no artificial limit or if the FPS limit is higher than what the hardware is capabable of rendering, for example on a large display.

[kovidgoyal]

frame rendering is artificially throttled in kitty (configurable via kitty.conf), precisely because I care about power usage more than FPS, see https://sw.kovidgoyal.net/kitty/performance.html

And even if it were not, frame render time would be a pretty poor indicator of performance anyway. Most of the time would be taken up by parsing the input from the child process and updating the terminal state machine according to it, not actually rendering frames.

Since kitty uses indirect rendering, a frame render involves simply sending a few KB of terminal state data to the GPU via memory mapped I/O and then running the two shaders on the GPU.

I dont know why alacritty choses to measure frame render time, seems like a pretty silly benchmark to me. A more suitable one (if you care about terminal performance in the sense of speed) would be to measure the time from when input is received to when it is displayed on screen. This however, is a much harder benchmark to construct.