WIP: Improve rendering performance

[ghost]

Fixes #905. Texture atlas generator is complete. It combines multiple images together to create a single image.

See #1061

[Samake]

Nice to see starts working on this. Thumbs up! What do you think which size atlas can handle later, i mean to hold the whole textures wouldnt be possible. Would it support multiple Atlasses by categories or so?

[Einheit-101]

Wow, Sam@ke alive? Nice.

@Topic: I would like to make some performance measurements. I have a lot of time recently.

[ghost]

What do you think which size atlas can handle later, i mean to hold the whole textures wouldnt be possible. Would it support multiple Atlasses by categories or so?

You simply need to supply the textures, it will automatically take care of sizes. So basically, nothing will change for scripters except for performance which will make a big difference. Maximum size for a single atlas is 16384x16384. If the textures cannot fit into one atlas then another will be created, so it's up to the program to decide how many atlases are needed, and it will generate them without any issue.

@Topic: I would like to make some performance measurements. I have a lot of time recently.

Sure, once I'm done with the PR. I'll let you know. :P

[Citizen01]

You have all the support I can give, just wanted to point out your PR includes 2 commits that are not related to this PR. You should have create a seperate branch on your side to prevent this.

[ghost]

You have all the support I can give, just wanted to point out your PR includes 2 commits that are not related to this PR. You should have create a seperate branch on your side to prevent this.

Yes, I realized it later. I'll squash and merge this PR, so these commits will disappear anyway ;p

[WADmitry]

How about externally prepared textures (using TexturePacker, etc)? For example, then I can use offsets between images and edge bleeding (to avoid alpha/interpolation artifacts on scaled textures).

Also, about maximum size of a single atlas. It depends on hardware. Some of "old" (or just low-end) hardware may not support 16K, 8K and even 4K textures. I think would be good add ability to adjust maximum size (16K/8K/4K).

[ghost]

How about externally prepared textures (using TexturePacker, etc)? For example, then I can use offsets between images and edge bleeding (to avoid alpha/interpolation artifacts on scaled textures).

If you're talking about rendering images, then you can always use dxDrawImageSection. I'm adding the atlas feature only to reduce the state changes (IDirect3DDevice9::SetTexture mainly) for 3D meshes.

Also, about maximum size of a single atlas. It depends on hardware. Some of "old" (or just low-end) hardware may not support 16K, 8K and even 4K textures. I think would be good add ability to adjust maximum size (16K/8K/4K).

Yes, we'll need to query the device capabilities and depending on that, we can set the maximum atlas size.

[WADmitry]

I'm adding the atlas feature only to reduce the state changes (IDirect3DDevice9::SetTexture mainly) for 3D meshes.

When textures joined into atlas then it may add graphical artifacts on edges (when a textures are independent then the problem usually is not appear) in some cases (for example, when texture is significally zoomed in, interpolation may pick texels of adjacent texture)

[ghost]

When textures joined into atlas then it may add graphical artifacts on edges (when a textures are independent then the problem usually is not appear) in some cases (for example, when texture is significally zoomed in, interpolation may pick texels of adjacent texture)

The program will automatically fix the issue for texture mipmaps and remap the 3D mesh UVs properly to avoid edge bleeding. Do you want to do something similar for 2D images that are within an atlas? If yes, then please explain it here.

[ghost]

@ArranTuna It's not just about performance. It also saves you lots of memory if a model is far away, GTA unloads it and shows the low LOD. Like I said, LOD models have their own TXD files. These TXD files will take up only a few KBs and save memory if you are not using custom models.

I do have plans to add LOD support for custom models after this PR.

@Einheit-101 Maybe you're right, but one can argue that if you are flying a hydra and you position your camera in a way where you see thousands of objects on your screen, there will be some performance issue on weak computers.

[ArranTuna]

I do have plans to add LOD support for custom models after this PR.

If you mean objects that servers have applied mods to, there is already some functions for that like setLowLODElement which I have used on modified land masses (but I just use the exact same texture instead of wasting the time and another model ID slot for a low quality texture).

[ghost]

I was able to generate a custom img file with atlas models. It was working, however, I could not load more than 4500 TXD files since that's the limit. The game kept crashing when I tried to exceed the limit, so I'll have to fix this by merging TXD files. I wanted to lift the model ID limit, but I got busy with this PR.

Just two more important things to actually benefit from the atlases:

• Rewrite some renderware rendering functions to support atlases
• Patch some GTA code to fix messed up color bug with vehicle models

[ghost]

All models seem to work, but merging TXD files is not a good idea. It results in TXD files being large. I limited the atlas TXD size to 2mb, and I still got 1200 atlas TXDs. I cannot have more than 1500 custom TXD files since GTA internal TXD names will take up the remaining 3500. When you are flying hydra, the streaming memory usage will be very high and objects will start to disappear.

It happens because loading a single DFF model and its TXD will result in unwanted textures being loaded along with the TXD, and it will eat lots of memory. That's not really what we are looking for. There are 8000 DFF models which are worth optimizing, so we need to uplift the limit to 12000 TXD files to make the streamer run smooth and use less memory. I'm going to use some code from fastman92 limit adjuster to make sure every DFF with atlas can have its own TXD.

[ghost]

I've decided to stop working on this, and go for DX11 instead. It's going to take a lot of work to implement the limit adjuster and fix the bugs/crashes that might appear. Moreover, the atlases textures will eat some good amount of RAM, and the only way to fix that is to reduce the texture sizes, but that's not what we want. Like I said, too much work to make it happen, why not spend that time on implementing upgrade to DX11? 🤔🤔🤔🤔🤔🤔

The atlases code is completely useless now. However, there is another feature in this PR, and that is the ability to output COL, DFF, and TXD files. Basically, we can add some new functions to allow scripters to create their own ingame editors for modifying DFF clumps, textures, and COL models, and then output them to a file.

Let's talk in #1061