Support legacy DDS format?

[superdump]

It seems all the DDS files I have do not support the DX10 extension and so I cannot use them with kram. Do you have any plans to add support for what I believe is the 'legacy' DDS format without the DX10 extension?

[alecazam]

I don’t really want to. The dx9 form of dds requires all sorts of fourCC inference, where dx10 has a dxgiformat and other unambiguous state. Also many formats don’t even have a fourCC code.

I’m trying to encourage a move away from png, jpeg, and dds to ktx2, but many Windows/console tools only take dds which has no metadata, and needs sidecar files. But metadata sidecar files don’t work with sandboxing unless you drag-drop the parent folder.

It’s a tiny amount of code to add for dx9, but could you just rewrite your dds as dx10 files. DX9 is over 20 years old. The references are in the dds code, and there’s already dds10 save support.

I did add an update to strip and replace the sRGB block in png and save the png out. Adobe can only seem to write out iccp/gama/chrm. Maybe something similar to update dx9 dds and save as dx10.

The problem is these dds9 or bad srgb png linger. Even my own source png had invalid srgb state. I want a corpus of correct image data.

[superdump]

I agree with you. I didn’t create the dds files but I would like to convert them to ktx2 and glTF. They are from standard, freely-available assets like Sponza, FlightHelmet, Bistro, etc. I just searched for all dds files I have and tried to run kram info on them and every single one gave me the error about only DX10 dds files being supported. Given these freely-available assets are commonly used for research papers and testing renderer code in engines or otherwise, it still feels useful to have some way to convert them, even if one has to provide the metadata that can be expressed in ktx2. Personally I see the need, but I can understand if you don't. It just makes it more awkward to get files converted to ktx2 which then maybe makes people just leave them as dds instead and dds lives on.

[alecazam]

I’ll add loading. You can then save out to ktx2 or ktx or dds10. kramv is already a viewer and thumbnailer, so it should be able to load these. What formats are you seeing?

[alecazam]

Also DDS is mip ordered unlike kTX/KTX2, but kram can read without deciding the BC data, and rewrite the blocks.

[alecazam]

Also note that gltf only supports jpg (useless), png (nearly useless), and KTX2. But the only compressed KTX2 supported uses basis. But KTX2 supports bc/etc/astc.

[superdump]

Also note that gltf only supports jpg (useless), png (nearly useless), and KTX2. But the only compressed KTX2 supported uses basis. But KTX2 supports bc/etc/astc.

Oh, interesting. I hadn’t seen anything about KTX2 in glTF only technically supporting ETC1S/UASTC. I’ll have to look that up. That sounds like a pretty harsh restriction and I wonder if anyone will adhere to it in practice given the broad support of formats in KTX2.

I think dds can be officially supported in glTF by an extension, albeit an MS-namespaced one: https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Vendor/MSFT_texture_dds/README.md

[alecazam]

Read support for dds with dx9-style data should be in now. Thumbnails and viewing should work, but it's untested. I don't have any of these files. I didn't tag it yet as a release.

No dds would only be compressed as BC if used with glTF, and that wouldn't be a universal format that anyone could open. Even KTX2 was slow out of the gate. But BC is off patent, Intel dropped their desktop ASTC support, and Apple added BC support to their ARM chip on the M1. So BC may return to being the universal compressed format, since I don't see desktops or consoles moving off it. I'm just tired of glTF files using jpg on normals, and having ringing everywhere. SketchFab was doing this on all their models to save transmission and storage, but it looks awful.

kramv already supports glTF2 and can load models and images from dds/ktx/ktx2 files. I just don't have a great 3d camera model in the version that is in github, so it's mostly not usable right now. There are default 3d shapes though.

Also kramv already has to decode ASTC/ETC files on Intel to display them. The gpus there don't support the format.

Basis is just storing a block format that's easily transcoded to others, and it's really a great idea. ETC1S is tiny on disk but the encode times were slow, and artifacts from it's ETC encode weren't shippable for my games. And the UASTC format wasn't any smaller and was much less flexible than shipping gpu block formats. You'll just need to first decompress it, and then transpile it. So that's two hops to the staging buffer instead of one for zstd/zlib decompress to the staging bufffer.

Also ASTC decodes to rgba16f and pollutes the texture cache. There's some extension to disable that behavior, but it was done in case hdr and ldr blocks were mixed, since the original spec didn't format distinguish hdr from ldr, even though some vendors split the format.

I do probably still need to add it to the "fixup" command. But right now that just loads and saves png. Also note that DX9 DDS files don't store srgb formats, 2d or cube arrays. You can still do an encode, and it will save out as KTX, but I don't think that will override the srgb format. Another user had asked to load BC from DDS and then save it to KTX/KTX2 without decoding the blocks, so I did that.

[superdump]

I've been using the binaries so I'll have to clone and build locally to try it out. Thanks for the quick turnaround!

Random note: when I used to work on FFmpeg there was a clear idea of trying to support whatever formats various other software could create, including dealing with their bugs, because once files are made and are out in the wild, something needs to be able to read them. But you can be opinionated and strict about what you write. It was highlighted to me that this is called the 'robustness principle': https://en.wikipedia.org/wiki/Robustness_principle

[alecazam]

I really don’t want to pull in jpg, tiff, ext, bmp, gif, etc. kram is meant to be agile and small. There are plenty of apps like image magick with obtuse cli args to handle the Swiss Army knife of image formats. Those can stay bloated with image load/save and color space handling. I chopped a bunch of formats out of the DDS loader that just need to go away, and have no equivalency on kram.

I’d prefer to focus on a few images and a few model formats. These are enough work. I’m trying to go deep in what can be done to generate, optimize, load, thumbnail, preview, and render those formats.

[alecazam]

I pushed a tag for it, but don't have time to test it. Let me know if that opens some/all of your files. There's a special path on dds input with encode, where it doesn't decompress the blocks, and can then rewrite as dds10/ktx/ktx2 (no zstd/zlib). So that should help with conversion. When passed ktx/ktx2, the encode path does decode the blocks. So I should probably formalize this mechanism in "fixup".