Alternative Image Loading Library for increased speed and additional formalts

[Looooong]

Texture2D.LoadImage is somewhat slow for large textures (greater than 2K) and it blocks the Unity main thread for the duration between dozens of milliseconds to a hundred milliseconds when loading these textures.

I suggest implementing asynchronous texture loader like unity-async-textureimport. This repo decodes the image in a separate task using FreeImage, which is the same library used by Unity to decode image data. Then it uses Texture2D.LoadRawTextureData on the Unity main thread to load the texture data. It creates a smoother exprience by reducing CPU lag spikes when loading textures at runtime.

I can contribute a PR for this feature.

[atteneder]

Hi @Looooong ,

Thanks for pointing out this repo.

I agree, Texture2D.LoadImage is very slow and I thought about similar solutions in the past. I didn't prioritize it yet, because imho KTX is the better way to go (if you're in control of your assets).

Ultimately Unity should provide a better API, but until this is not reality, I would welcome a PR like that. I have some considerations though:

• I have to investigate if Unity's Jpeg/PNG decoders can be used on a thread in some way
• Bundling the entire FreeImage library seems excessive (10MB DLL). Isn't there a smaller alternative for Jpeg/PNG only?
• We would need more than just Windows and Linux binaries
• I'd prefer a UPM package as optional dependency. This way users can choose to use it (or not)

Kind regards,
Andi

[Looooong]

We are developing a platform on Unity that allow users to import their own assets. So I can say that we don't have 100% control of the assets uploaded to our platforms. That's why we invested in asynchronous texture importer for our platform. It is used not only for glTF but for other formats as well.

Right now, since we already have the code here, I think that we can just integrate this library real quick before investigating in another better solution later on. I can convert unity-async-textureimport into a separate UPM easily.

[Looooong]

@atteneder I have published UnityAsyncImageLoader. So the next step is to expose an interface in glTF to use this package. Is that correct?

[Looooong]

• Bundling the entire FreeImage library seems excessive (10MB DLL). Isn't there a smaller alternative for Jpeg/PNG only?

Btw, I want to point out that FreeImage supports more than 30 different image formats, not just JPEG and PNG. On the other hand, I agree that the library packs more utilities than we ever need.

[atteneder]

Question is: How useful is support for 30 image formats? If your app requires them (outside of glTF) anyways, then yes, but glTF supports only PNG and Jpeg (and WebP via extensions). It would be a bad idea to create non-compliant glTF files with unsupported formats, so 27/28 formats would not ever be used. Besides, opposing to KTX/BasisU all of those formats are not natively supported by GPUs.

Note: I'm currently on paternal leave and will only answer sporadically. I'll be able to provide help later this month.

[Looooong]

I didn't notice that glTF only supports a limited set of image formats. If that's the case, I have another alternative here: SAIL. Also, I found a list of image libraries on OpenGL wiki as well.

[PHORIA-Phill]

Would love to see this tackled!

[atteneder]

An idea I wanted to document:

For WebGL we could leverage the browser's capabilities to decode PNG/JPGs in Javascript and bridge it back to Unity. I've heard browsers are good at this.

[PHORIA-Phill]

Is it possible to revisit this enhancement? It'd be really helpful to be able to import user-submitted GLBs without the extreme impact of Texture2D.LoadImage on the frame time.

[Blackclaws]

I'd like to point out that FreeImage is GPLv3/GPLv2 or FreeImageLicense in licensing. This might be problematic for some so this enhancement, if it ultimately falls back on FreeImage should be behind a flag.

In general I'd say it would be easiest if you'd expose methods on the GltfImporter that allow configuring image converters for each image type, falling back to Texture2D.LoadImage if none are configured. That way it would be easy enough to inject any converter and there wouldn't be any dependency on a particular library.

Edit: Alright, seeing as how Unity itself uses FreeImage apparently and it does so under the FreeImage Public license this shouldn't be a problem after all. I still stand by my comment that it should be easily configurable from a user perspective as they might want to use a specific library and only for one or two formats, not a whole suite for 30+ formats.

[havalbon]

Has any traction been made on resolving this issue?

We are attempting to transition from UnityGLTF to GLTFast for our 3d object load and render pipeline. Unfortunately, not having a way to either offload the image decoding onto a background thread using a native plugin and/or configure/override the image conversion is a blocker for us on HoloLens.

I understand that the recommended approach is to use KTX/Basis Universal compressed textures. However, our flow involves runtime loading of user-generated content uploaded to a database, which means that we would need to also implement transcoding of .jpeg and .png textures at upload time.

Additionally, are there any current or future plans to support .dds compressed textures?