Graphic Rendering Roadmap

There are a number of enhancements that must be made to the graphics rendering API to more easily support some lacking features and keep our sanity. This page is an attempt to iterate the required changes and capture some of the reasons they ought to be undertaken.

One of the first things that must be done is to wrangle control of SDLs texture shader. Fighting with it is a bottleneck, a source of bugs (#1377), and limiting our path for desired features (#1098). We can do this at driver start up by rendering the scratchbuffer to the screen and the fetching the current program from OpenGL, detaching the shaders, attaching our own shaders, then relinking the program. Once this is done we will be able to carve a path for the shader to be the totality of our rendering and won't depend on any of the sprite "versioning" tricks.

Refactoring palettes would probably be the biggest effort, but also the greatest impact. Palettes are the main driving force for the extra texture caching ("versioning") we do for sprites. This is both a lot of extra memory and a bottleneck from updating GPU textures. We cannot and should not revert to our old OpenGL palette shader as it was horribly inefficient (lots of small sprites) and complex due to being grafted on to the now dead and buried original video driver. Instead, we should create a PaletteManager to consolidate all palettes into a single 256x? image so that we only have one (or possibly a few) GPU texture that we can later bind to some constant predetermined texture unit.

Once we have consolidated Palettes and gotten control over the texture shader program we can extend our shader to support palette lookups and shading. There are a few minor, yet annoying, additional prerequisites to this.

1. We would have to somehow unify VideoBuffer and Sprite2D. They both are fundamentally SDL_Textures.
2. We would have to write our own 8-bit SDL_Surface -> SDL_Texture converter to cram the "pixels" into the red channel of the SDL_Texture.
3. probably need to consolidate our Rendering API under 1 or 2 functions with some kind of RenderingContext param to control clipping/srcrect/destrect/flags/colors
4. probably something I haven't thought of yet.

The last large effort task is to extract the GlyphAtlas from Font and make it generic so that animations use a single Atlas (so single/few GPU texture). This also requires a bit of a refactor to Sprite so that we can have disparate sprites referencing different parts of the same backing storage. This is mainly to achieve much better performance.

There are some lower effort work that depends on some of the above:

1. OpenGLES2 support: depends on gaining control of the SDL_Renderer texture shader.
2. Zoom in/out on the game: depends on refactoring the Blitting API to include RenderingContext so we can scale source/dest. Also depends on removal of the "versioning" because we cannot be software scaling anything (too much effort, too poor performance).