Aspect ratio and viewport improvements

[johnnovak]

Description

Let's start with the release notes!

---

Aspect ratio and viewport improvements

Stretching the image to completely fill the screen has been an often requested feature (e.g., to play text adventures without black bars on the sides). Now we're giving you not only that but an entire new mechanism to apply arbitrary aspect ratios, zoom into the DOS content, and even emulate the horizontal and vertical stretch controls of CRT monitors of yore! 😎

The key element of the feature is the new aspect = stretch mode which stretches the image to the extents of the viewport. For example, to make a game completely fill the screen, use the following config:

[sdl]
fullscreen = on

[render]
aspect = stretch
viewport = fit
integer_scaling = off

The second piece of the puzzle is the new relative mode added to the viewport setting (viewport_resolution has been renamed to viewport and moved to the [render] section). In relative mode, the viewport is a 4:3 aspect ratio rectangle fit into the window or screen as the starting point, which is then scaled by the specified horizontal and vertical stretch factors. The resulting viewport is allowed to extend beyond the edges of the window or the screen, so this can be used to "zoom" into the image while forcing arbitrary aspect ratios.

For example, you can aspect ratio correct lazy Hercules conversions that just reused the EGA/VGA assets and make them fill the screen better by zooming in a little. The following example illustrates this on Prince of Persia in Hercules mode:

[render]
aspect = stretch
viewport = relative 110% 160%
integer_scaling = off

Captions:

1. Prince of Persia without any custom stretching
2. Prince of Persia with aspect = stretch and viewport = relative 110% 160%

You can also use the "Strech Axis", "Inc Stretch", and "Dec Stretch" hotkey actions (unbinded by default) to adjust the horizontal and vertical stretch factors in real-time, just like on a real CRT monitor. The current viewport settings are logged, so you can simply copy them into your config.

Note that using 'relative' mode with 'integer_scaling' enabled could lead to surprising (but correct) results. It is recommended to disable integer scaling while you're still getting to grips with 'relative' mode.

---

Technical notes

This PR extends the aspect and viewport settings to allow for the below advanced use cases:

• Stretch DOS content to fullscreen to get rid of the left & right black bars on 16:9 flat panels (requested here, by a guy on Discord who had a medical reason for this as the black bars caused him discomfort, and by a couple of people on various forums)

• "Zoom into" the DOS content while keeping the correct aspect ratio (requested here)

• Apply arbitrary aspect ratios manually (the "anything goes" option—might be handy in some rare situations as this effetively emulates the vsync/hsync controls on a CRT monitor; this was requested for some pinball games here)

• Apply arbitrary aspect ratios and "zoom into" the DOS content at the same time (the previous two features combined; on a real CRT, you could enlarge the image so much with the horiz/vert size controls that parts of the image got cut off. This is useful for Hercules games that simply reuse the 320x200 assets in the 720x348 Hercules graphics mode; these games appear overly squashed with authentic Hercules emulation and the graphics don't fill the screen horizontally either (more info))

I wasn't a fan of such requests initially because I thought they would complicate things too much and it would be too hard or even impossible to make them work consistently and logically with our existing features, but I found a way to add support for them cleanly with a minimum set of extra options. These enhancements/changes are:

• 100% backward compatible
• Don't complicate the normal use cases in any way
• Work with integer scaling as one would expect (integer scaling is always aspect-preserving, period)
• Work with upscaled screenshots
• Will work with the planned overscan/border support feature
• Will work with any future video capture enhancements as well (because the images passed to image/video capturer will be simply just tagged with an arbitrary aspect ratio, so things should JustWork(tm))
• We could even implement horiz/vert size control sliders in the OSD to set the CRT-like image stretching in real-time! 😎

Changes

Please see the commit messages for the detailed list of changes & further details and justifications, but here are the main things:

• aspect setting: introduce auto and square-pixels aliases for on and off, respectively
• viewport_resolution: rename to viewport and move it to render
• Introduce new aspect stretch mode (as per above)
• Introduce new relative mode to the viewport setting (as per above)
• Tons of SDL main cleanup & refactoring stuff
• Lots of log warning cleanup & normalisation
• Improve some config descriptions

Related issues

A completely full-screen or filled screen option
#1474

Can I enlarge the screen content from the centre?
#2958

Graphical issues with some Pinball games
#1578

Manual testing

• Tested error handling when providing various invalid aspect and viewport settings.
• Tested all aspect and viewport combinations when set from the config, and at runtime in windowed mode (also while resizing the window—this is important!) and in fullscreen. Tested both opengl and texture output modes.
• Tested that integer scaling still works as expected (might be counterintuitive a bit; this is advanced mode, be warned)
• Confirmed that upscaled image capture works as expected and captures the image in the custom aspect ratio in relative mode & the image metadata gets written correctly.
• Checked that the render dimensions get logged as they should.
• Regression-tested normal and seamless mouse modes with all setting permutations in DOS Navigator and Win 3.x with the VMware mouse drivers.
• Tested on both macOS and Win 10.

Future work

• Add support for separate scale factors for text and graphics modes (because if you set up zoom for a Hercules game, the console becomes unusable as half of it is cut off). Proposed format: viewport = relative text 100% 100% graphics 110% 160%
• Consider adding yet another aspect mode called auto-and-stretch that uses the result of the aspect = auto mode for the further horiz/vert stretching (which is based on VGA timings, not just simply using a 4:3 rectangle as starting point as the current relative mode does).
• The stretch hotkeys make the image flicker a little, but improving that would be a nightmare with the current code organisation. That's something to do after the SDL/rendering refactor.
• Real-time OSD stretch controls, of course—one day 😎 🤘🏻

Checklist

Please tick the items as you have addressed them. Don't remove items; leave the ones that are not applicable unchecked.

I have:

• followed the project's contributing guidelines and code of conduct.
• performed a self-review of my code.
• commented on the particularly hard-to-understand areas of my code.
• split my work into well-defined, bisectable commits, and I named my commits well.
• applied the appropriate labels (bug, enhancement, refactoring, documentation, etc.)
• checked that all my commits can be built.
• confirmed that my code does not cause performance regressions (e.g., by running the Quake benchmark).
• added unit tests where applicable to prove the correctness of my code and to avoid future regressions.
• made corresponding changes to the documentation or the website according to the documentation guidelines.
• locally verified my website or documentation changes.

[kcgen]

Mostly minor suggestions.
Lots of nice improvements in this, @johnnovak !

[johnnovak]

Thanks for the early review @kcgen , just checking that you're aware it's still a draft and work-in-progress? It's only about 70% finished yet. There's gonna be lots of force pushes and re-jigging things still, plus not everything in the top comment is implemented yet (and that comment is not finished either; there will be screenshots coming, etc.)

[kcgen]

Thanks for the early review @kcgen , just checking that you're aware it's still a draft and work-in-progress? It's only about 70% finished yet. There's gonna be lots of force pushes and re-jigging things still, plus not everything in the top comment is implemented yet (and that comment is not finished either; there will be screenshots coming, etc.)

Yup - might as well nip the easy stuff early. Was only commenting on superficial/rote things.

[kcgen]

Oh - wanted to mention this early on - it would be really slick to combine the Fraction class with aspect to allow for natural strings like 16:9, 4:3, 4:5 (apologies if it's already there and I just didn't catch it! 😅)

[johnnovak]

Oh - wanted to mention this early on - it would be really slick to combine the Fraction class with aspect to allow for natural strings like 16:9, 4:3, 4:5 (apologies if it's already there and I just didn't catch it! 😅)

Cool idea, that's something to think about, although in its current iteration things will work a bit differently. You'll see when I'm done, then we can discuss again 👍🏻 Maybe it could be an additional enhancement, yeah, but I'll finish first what I had in mind 😄

[Kappa971]

Oh - wanted to mention this early on - it would be really slick to combine the Fraction class with aspect to allow for natural strings like 16:9, 4:3, 4:5 (apologies if it's already there and I just didn't catch it! 😅)

Speaking as a normal user, this is an interesting addition but the overflow option seems a bit too complex for an inexpert user (well maybe even for an expert user).
Wouldn't an option like aspect = 16:9 <zoom%> be simpler? (with aspect = 4:3 as default).
I don't know...

[johnnovak]

Oh - wanted to mention this early on - it would be really slick to combine the Fraction class with aspect to allow for natural strings like 16:9, 4:3, 4:5 (apologies if it's already there and I just didn't catch it! 😅)

Speaking as a normal user, this is an interesting addition but the overflow option seems a bit too complex for an inexpert user (well maybe even for an expert user). Wouldn't an option like aspect = 16:9 <zoom%> be simpler? (with aspect = 4:3 as default). I don't know...

Good to see you're thinking about possible simplifications and taking user friendliness into account @kcgen @Kappa971!

To be absolutely crystal clear, I'd also like to make this as simple and intuitive as I can, so I gave your ideas very serious consideration, which means literally spending the last 3 hours (!) sketching out ideas on alternative config implementations based on your suggestions and trying out various things hacking the code. The TL;DR is, I've been going around in circles because the simplifications all break down in some scenarios and/or are unsupportable or rather meaningless. So I'm pretty sure I've hit a local maximum with my current solution that balances flexibility, simplicity, and ease of use. I just can't simplify it any further while catering for all current and foreseeable future needs, and the current solution is already a culmination of a week's thinking, carefully weighing of pros and cons of various possible solutions. This is already about my 4th or 5th iteration of the config options!

Believe me, alternative solutions would make things a lot more complex. I've spent about 100+ hours dealing with aspect ratio-related things earlier this year dealing with the VGA code. One side effect of that work is the following huge aspect ratio table of various DOS video modes:
https://github.com/dosbox-staging/dosbox-staging/wiki/Video-emulation-tests#summary-of-dos-video-modes

I don't want to appear condescending and I'm absolutely saying this in a friendly, amicable manner: understanding how the various rendering options work together with aspect ratio stuff requires a bit of an expert knowledge — it's far more complex than just "yeah, stretch to 4:3 or 16:9", that's not even scratching the surface of understanding the implications and what we're trying to solve here. Just to enter a meaningful discussion about the issue I'm trying to solve, a deep understanding of the below points is required:

• 100% understand the differences between storage, display, and pixel aspect ratios,
• the reasons why graphics in some DOS games appear in the wrong aspect ratio (e.g., Amiga ports, Hercules games, etc.; these reasons are as much "societal" as technical),
• familiarity how wildly different pixel aspect ratios all pre-640x480 VGA / square-pixel VESA/SVGA video modes have (see my table)
• how CRT monitors work, specifically the horiz and vert size adjustments (how early monitors did not have these controls at all, then later monitors had "global" analog controls, then even later 90s monitors had digital controls where you could dial in specific horiz/vert stretch settings per display timings, so different stretch factors per video mode (!) in practice)
• what our host-side limitations on modern flat screens are,
• how integer scaling, shaders, existing viewport restriction etc. would work together with all this (including border support),
• how would all this be adjusted in the OSD in real-time in an intuitive manner,
• and I'm sure there's something else I forgot...

I could spend the whole day writing a dissertation about how some of the proposed simplifications would break various things (they are good ideas on the surface, but naive), but just a quick rundown:

• aspect = on does not simply stretch whatever is rendered on the DOS side to a 4:3 output rectangle, that's a misconception. The actual pixel aspect ratio (PAR) is calculated based on VGA timings, then based on the resulting PAR and video mode timings we can end up with pretty much any output image aspect ratio. It is usually 4:3, or close to 4:3, but it can be widescreen(ish) in some games and VESA text modes. Stretching to 4:3 no matter what would break a small but significant number of games/programs.
• aspect = 16:9 is not that helpful; the use case here would be for people who just wanna stretch the content to fullscreen, no matter what, to get rid of any black bars. They don't care if they have a 16:9 or a 16:10 screen, or whatever else; for them, aspect = fit and the default viewport_resolution = fit option is actually best for the job.
• To make Hercules games displaying 320x200 assets in the 720x348 Hercules graphics to not appear squashed, coming up with the correct fractional (!) aspect ratio would do your head in, trust me. I just gave up after literally 30 mins of trying and failing 😅 Surely it can be calculated, but that's not helpful to users. What is just allowing them set arbitraty stretch factors as percentages, just like on a real CRT with the horiz/vert adjustments. Then you adjust until it looks right, done. This could be exposed on the OSD too as continuous stretch controls. Calculating this exactly then having to specify aspect = 13390:10004 or aspect = 0.898441 (just making shit up, but it would be something obscure like this) is not more user-friendly.
• There is the dilemma of what the aspect setting should actually do: correcting pixel aspect ratios, or stretching the output to an arbitrary output image aspect ratio (display aspect ratio)? Currently, it's a bit vague, but it's best to keep it that way, actually: on now does the calculation based on VGA timings (and in some cases heuristics), off just assumes square pixels. I will rename the aspect = viewport setting to aspect = fit which is more intuitive; it will just massage the aspect ratios so the image fits into the viewport rectangle, no matter what. Specifying PARs would make things quite complex, and PARs are different for pretty much every non-square-pixel resolution—let's not even go there.
• Now that we've established that custom "stretch factors" are the way to go, the simplest way to implement that is to allow arbitrary viewport sizes and let the DOS content be simply stretched into the viewport rectangle. That's it! Doesn't get much simpler than that! That's what the aspect = fit and viewport_resolution = relative 248.89x140% will do (I'm renaming the somewhat confusing overflow to relative as the percent-sizes are always relative to the canvas height). Moreover, this will be supportable in the OSD simply and intuitively: select the "Fit" option from the "Aspect" drop-down, then move the two "Viewport size" sliders and see the changes in real-time! Yay! 🎉

---

I could go on...

In the end:

• Complex issues can be simplified to some extent, but can't be made simpler at a certain point without making the feature useless.
• To use complex features, you either need to have a full understanding of what's involved (I 100% agree that will be rare among users), OR you need to rely on "config recipes" for the most common things you wanna do.
• These recipes will be provided in the manual—that's the EasyMode(tm) where person X just wants to solve a practical problem with minimum fuss. Wanna do this, put this into the config. The manual testing section will contain such "recipes" in action.
• Advanced users who understand these details will have total freedom and can do whatever.

Now, after a 4-hour detour, finishing the feature as I had planned 😎

[johnnovak]

Ok, this is now 100% complete 😅 I've made the Rect class industrial strength, incl. test coverage @kcgen 😎

6126fd5