SDL3 logistics

[slouken]

So we're planning to start SDL3 development after the next patch release, and this thread is to discuss and document logistics of how we maintain SDL2 and continue SDL3 development.

Our current plan is to make all the ABI breaking changes immediately before the very first SDL3 release, so we have a stable ABI from the very beginning. I'd also like to simplify and remove some of the APIs that are not widely used and awkwardly implemented. We can re-add them as needed, but it would be good to start from a simpler API space if possible.

One thing that will be important is that we have sdl2-compat from the very beginning, so we'll have that built into the SDL3 source tree. At that point we'll also move sdl12-compat into the tree as well, so we can do releases and regression testing all the way from SDL 1.2 -> SDL 3.0

Questions for @libsdl-org/a-team and @smcv:

• Do we create a branch for SDL2 or for SDL3? i.e. what is main?
• How do we reasonably merge SDL2 changes to SDL3 as development and bugfixes continue?

Once we figure out logistics, we can open a set of bugs for details on SDL3 changes from our suggestions thread.

[1bsyl]

It seems to me there are almost daily commit to SDL2, even without new feature (bug fix, doc, build system, versions of dependencies, API deprecated, etc.). So if we branch out now for SDL3, we'll have to constantly re-apply fixes to SDL-2 , SDL-3, (and SDL-2.24.x).

Maybe we can start to have real PR implemented for SDL3, and when we have a consequent number of PR, we branch for SDL3.
The advantage of PR is that github constinuously check for automatic merge of not. Also we can see if SDL2 commits calm down during this time.

Btw, there were a suggestion about SDL3 to have the API renamed. If you want this, for this one, maybe we should not do a PR, but provide a script to rename all API. So that we can use in other project SDL_image, mix, ttf, and personal projects.
can be down with sed. If we do more complicated stuff, like we changing parameters order across several lines, we could use semantic patch (like https://coccinelle.gitlabpages.inria.fr/website/ ).

Not sure if we should use SDL_DEPRECATED to keep some old api in SDL3 ?

[slouken]

I agree about the API renaming script, that sounds handy.

We should remove all the functions marked deprecated and not deprecate any new ones, we should just remove them if we don't want them in the new API. Deprecation should be removal on an ABI break.

[smcv]

GTK 3 to GTK 4 is an example of a recent intentional API/ABI break that I think was handled reasonably well, which might act as a useful reference. One thing that I think was really helpful is that there is a porting guide in the documentation.

From the API/ABI breaks I've seen, the biggest risk seems to be that the new version is constantly delayed by "just one more incompatible change..." and either takes a long time (during which time developers' focus is split between the old and new side of the break), or never actually ships as stable. GTK took more than 3 years to get from 3.91.2 to 4.0.0, and during some of that time GTK 3 was in a weird state where it was claimed to be in bugfix-only maintenance mode, but actually gained new features, some of which caused regressions and reduced downstream distributors' confidence in being able to ship bugfix-only point releases in their stable distributions.

I think one way to mitigate that would be to plan out what the breaking changes are going to be, do them all relatively early, ship a first prerelease, and then be relatively reluctant to make additional breaking changes between that first prerelease, subsequent prereleases, and 3.0.0.

Another way to mitigate that would be to gradually raise the bar for how big a new feature can be and still be considered for inclusion in SDL 2.x, with larger or more intrusive new features being "SDL 3 or nothing", creating an incentive to get SDL 3 out of the door rather than continuing with SDL 2.

For SDL specifically, one bit of timing that I'm concerned about is that #6362 is trying to make native Wayland the default (which involves a relatively large amount of code churn), at the same time that #6327 is doing a major-version break. What does the timeline for those changes look like? It might make sense to set a deadline for native Wayland being the default in SDL 2, so that if the maintainers think native Wayland by default is ready before the deadline, it can ship, but if it isn't ready by that time then SDL 2 will never default to native Wayland. Or, alternatively, if the timeline for SDL 3 is intended to be relatively short, it might make sense to have native Wayland preferred over X11 in SDL 3 from the beginning, but say that SDL 2 is never going to swap its default.

Do we create a branch for SDL2 or for SDL3? i.e. what is main?

I think main should be SDL 3, with SDL 2 feature work (if any) relegated to a non-default branch (2.x or something). Otherwise, there will be a temptation to develop new features against SDL 2 and forward-port them to SDL 3 afterwards, resulting in changes flowing in both directions and some of them getting lost.

Our current plan is to make all the ABI breaking changes immediately before the very first SDL3 release, so we have a stable ABI from the very beginning

I think in practice we are going to need prereleases of SDL3, perhaps marked as SDL3 version 2.99.1, 2.99.2..., with the option to make small ABI/API breaks between prereleases - similar to the way new API introduced in 2.25.x is "at risk" until 2.26.0, but applying to the entire library and not just new things (although, again, it would be good to get the majority of planned breaks done before the first prerelease). Taking random snapshots from git is all very well, but prereleases seem like a good way to focus testing onto a smaller number of non-moving targets.

In GTK, they deliberately used a different SONAME (machine-readable library name) between prereleases and the stable release: on Linux, the prereleases were libgtk-4.so.0 and the stable release is libgtk-4.so.1, to have a clean break and force prerelease-dependent applications to be recompiled against the stable release. That might be a good idea here too.

One thing that will be important is that we have sdl2-compat from the very beginning, so we'll have that built into the SDL3 source tree. At that point we'll also move sdl12-compat into the tree as well, so we can do releases and regression testing all the way from SDL 1.2 -> SDL 3.0

I like the idea of having SDL 3 contain sdl2-compat, so that we can use SDL 2 games (and indeed bindings like pysdl2) to get ready-made test-cases for SDL 3.

However, I'm a little concerned that if sdl12-compat and sdl2-compat are in-scope and in-tree, that will drag out the transition by meaning that SDL 3.0.0 won't be released until the long tail of weird/old SDL 1.2 and SDL 2 games have all been tested and found to work. Perhaps sdl12-compat should not be in-tree immediately, but should move into the tree later, so that it doesn't hold up SDL 3.0.0?

An in-tree sdl12-compat will have to identify itself as a (probably hard-coded and fixed) version number that is higher than any version of SDL 1.2 or the separate sdl12-compat, perhaps 1.2.80 or something. Similarly, an in-tree sdl2-compat will have to identify itself as a version number that is higher than any real SDL 2 release.

[slouken]

I think as soon as we branch 2.x, all feature work stops for 2.0, except for critical changes for shipping products.

The Wayland transition should wait until SDL3, and I updated #6362.

I think the idea is that we can ship sdl12-compat and sdl2-compat releases separately from SDL3 releases, tagged appropriately. Maybe there isn't value in keeping them in-tree?

I agree that SDL3 is going to need a series of prereleases, but they shouldn't be versioned 2.X, otherwise SDL2 applications might try to link with them. They should use the SDL3 soname (and SDL3.dll, etc.) but will be marked as pre-release on the website. Maybe we need the first prereleases to be 3.1.X and the first "stable" release to be 3.2.0?

Thanks for the feedback!

[smcv]

I agree that SDL3 is going to need a series of prereleases, but they shouldn't be versioned 2.X, otherwise SDL2 applications might try to link with them. They should use the SDL3 soname (and SDL3.dll, etc.) but will be marked as pre-release on the website.

What I was expecting was that they'd have 3 in all of the library's names - libSDL3.so.0, SDL3.dll, pkg-config sdl3.pc or SDL3.pc, CMake SDL3::SDL3, etc. - but with SDL_GetVersion() etc. initially returning something like (2, 99, 1). The way this worked in GTK 4 was that prerelease 3.96.0 looked like libgtk-4.so.0 -> libgtk-4.so.0.9600.0, and then the first stable release 4.0.0 was libgtk-4.so.1 -> libgtk-4.so.1.0.0, and the current stable release 4.8.1 is libgtk-4.so.1 -> libgtk-4.so.1.800.1 (part of the reason the SONAME was bumped for the stable release was to have a clean break from prereleases to stable releases, but part of it was to stop the version in the filename from going backwards).

Or, if you want the major version number in prerelease tarball names, SDL_GetVersion(), etc. to always be 3 from the start, then yes I think you'll need to start from 3.1.x prereleases leading to a 3.2.0 stable release, with 3.0.0 never existing.

[smcv]

I think the idea is that we can ship sdl12-compat and sdl2-compat releases separately from SDL3 releases, tagged appropriately. Maybe there isn't value in keeping them in-tree?

If they are going to have their own independent version number and release schedule, then I think they should probably be a separate git repo.

Having them in-tree only really makes sense to me if the whole tree is released as a unit: in Debian and similar distributions, we'd handle that by having one source package for the whole tree, which we'd compile once and then split into binary packages (many distributions have one .deb or equivalent per shared library, so that you can install a subset).

[icculus]

Fwiw, some things that sdl12-compat had to deal with, that made it require intense engineering effort:

• Serious development started on it years and years later, long after SDL2 was shipped and even longer since 1.2 was treated as abandoned. We're really only now at the point where testing odd little games from the Debian archives makes sense (and has been extremely valuable!). Had we started earlier, we would have been at that point in 2015 and not 2022, though.
• It's trying to "virtualize" certain aspects of SDL 1.2 that aren't good fits for modern systems (software rendered games at low resolutions are now pushed onto the GPU and scaled, CD-ROM audio is emulated with directories full of MP3s, etc). It's totally possible that we're asking ourselves how to seamlessly get old SDL2-based binaries of Portal 2 to run on a VR headset in the next decade, but I think in the current era there is less struggle in this regard, between SDL2 and SDL3.
• SDL 1.2 didn't have a lot of discipline about API and ABI details, so we're still finding things that are expected to work in specific ways in specific scenarios.

The hope is that an sdl2-compat has significantly less tapdancing to do than the 1.2 compatibility layer--much more of the API passing through unmolested--so keeping it maintained in parallel, in-tree, will hurt less. And we can also dictate that API changes must have the appropriate sdl2-compat work in the same commit.

That being said, it's a good point that releases aren't likely to line up for long; it's possible sdl2-compat has a release or two and almost never needs another until SDL4, and the 1.2 layer will hopefully be reaching that point soon, too (and maybe won't need a lot of changes to retarget it to SDL3...?).

[orowith2os]

I think this is a good place to post my suggestion: for SDL3, possibly improve on the Wayland CSDs.

For example, right now, it's a bit weird, especially for applications, whereas games are less of an issue.
Thus, I suggest having a system that allows the client to draw its own CSD if it likes, and maybe have a preset list of options (for example, do a desktop environment check for GNOME, and if it passes, use a GNOME-like CSD. If anything else, like KDE, use a "generic" CSD like it is now, or something fitting of that DE if possible).

[sulix]

Thus, I suggest having a system that allows the client to draw its own CSD if it likes, and maybe have a preset list of options (for example, do a desktop environment check for GNOME, and if it passes, use a GNOME-like CSD. If anything else, like KDE, use a "generic" CSD like it is now, or something fitting of that DE if possible).

Apps can already draw their own CSD, right? Just pass SDL_WINDOW_NOBORDER, and then use SDL_SetWindowHitTest() to implement a title bar.

So, I guess what we really need is a way of checking to see if the system would prefer we draw our own decorations.

As for the preset CSD styles for apps which don't want to draw their own decorations, we outsource that to libdecor which is working on a Gnome/GTK style plugin. (I definitely don't think SDL wants to be in the business of imitating different platforms' styles if we can avoid it — providing a libdecor plugin or equivalent should be the platform/DE's job, IMO).

[orowith2os]

Replying to #6327 (comment)

Nice to see that it's possible to draw the CSDs manually, at least. I wasn't aware that was possible.

I guess the check is what I'm aiming for, yeah. I was aware that libdecor was in use for this, but thought the weird-looking CSD would be how it is for now.

So libdecor will handle which CSDs to draw, it just has to have the plugins for it? How does it handle choosing them? Is that up to SDL, or does libdecor handle everything? CSDs are foreign to me, especially with Wayland.

[icculus]

Currently libdecor checks if SSD is available and favors that. If not, it uses a plug-in. There's a pull request for actual GTK+/Gnome CSD, but it hasn't been merged into libdecor yet, which is why you get a basic black titlebar currently.

[flibitijibibo]

Quick question since it's on my mind: What's our process for backporting patches to the SDL2 branch? For example, #5906 will definitely want to be in both 2 and 3, should we just commit them as we see them or do we want to batch them similar to the stable branches?

[slouken]

Put it in main, and issue a separate pull request for SDL2 if you feel it's important enough.