[unity] Delayed on-demand loading of Atlas assets

[HaraldCsaszar]

Mentioned on the forum here:
http://esotericsoftware.com/forum/How-to-control-texture-memory-usage-for-skeleton-with-skins-13240
http://esotericsoftware.com/forum/I-tried-to-export-with-command-line-but-not-file-is-created-14258?start=25
Also requested on this forum thread:
http://esotericsoftware.com/forum/Memory-management-of-character-with-many-outfits-15867

In Unity all referenced assets are automatically loaded, so there should be a comfortable way provided that allows to:

1. Assign multiple atlas assets at the SkeletonDataAsset in the Unity editor and keep them as-is upon reimport, even when not all attachments are found. A preferences parameter shall allow disabling any warnings when not all attachments are found.
2. Load assigned atlas assets (textures) on demand when attachments are required of this atlas. Could be managed by attachment name prefix like green/ or tier4/. There should be a checkbox Load on-demand provided in the Inspector to enable on-demand loading.
3. Unloading of no longer used atlas textures shall be triggerable via code. Auto unload shall be configurable per atlas asset as an Inspector checkbox.

Implementation notes:
ad 1:

• SkeletonDataAsset already holds an array of atlas assets: AtlasAssetBase[] atlasAssets. Current logic just reports error messages when not all attachments are found and will then be reset upon reimport, losing assigned atlas assets.

ad 2:

• Attachment name prefix could be a customizable pattern string list, or common prefix automatically extracted from the atlas file.
• On-demand loading needs to allow for loading via asset bundles, resources folder, addressables, etc.
• Atlas asset references shall be displayed as assignable references in the Unity Inspector, but serialized as strings or other non-reference types upon creating the build, to avoid the direct link to the assets.

[jacattrongnlh]

Is there any plan to implement this soon? Our game is having memory issue related to this, it could really use this feature

[HaraldCsaszar]

@jacattrongnlh If you need a solution soon, we would highly recommend implementing a solution on your own in the meantime.

[HaraldCsaszar]

First of all, sorry for the long delay!
A first experimental version of an on-demand Addressables texture loader has now been implemented.

You can try it out using the following steps:

1. Downloading the latest spine-unity 4.1 unitypackage from the download page or updating your 4.1 spine-unity runtime via git to the latest revision. This runtime version provides basic integration of the abstract base class OnDemandTextureLoader from which on-demand loaders derive from.

2. Install the newly added UPM packages com.esotericsoftware.spine.on-demand-loading and com.esotericsoftware.spine.addressables.
   You can install them via the Unity Package Manager if you've installed the spine-unity runtime via the Package Manager as well, using the normal Add package from git URL .. button using the following git URLs:

https://github.com/EsotericSoftware/spine-runtimes.git?path=spine-unity/Modules/com.esotericsoftware.spine.on-demand-loading#4.1
https://github.com/EsotericSoftware/spine-runtimes.git?path=spine-unity/Modules/com.esotericsoftware.spine.addressables#4.1

If you've installed the spine-unity runtime via unitypackage, you need to download the two packages above and before adding them to your project remove the line "com.esotericsoftware.spine.spine-unity": "4.1.21" from both package.json files (removing the spine-unity UPM package dependency). More comfortable official zip packages will be released soon, this only serves as a first test package to don't keep you waiting longer than necessary.

For usage of the Addressables on demand loader package, please see the readme.md file in com.esotericsoftware.spine.addressables/Documentation.

Please share your thoughts on the forum, especially if you encounter any problems or have ideas for improvement.
Hope you like it! :)

[HaraldCsaszar]

On-demand (Addressables) texture loading now supports SkeletonGraphic.
OnDemandTextureLoader received an added method RequestLoadTexture and delegate TextureRequested.
Fixed missing loader asset name suffix.

New spine-unity 4.1 and 4.2-beta unitypackages are available for download:
https://esotericsoftware.com/spine-unity-download

You can update and try out the two UPM packages as described in the above comment.

As always, please let us know if you encounter any issues or have any ideas for improvement. :)

[Gamer-XP]

I needed something like this for the project, and decided to test it. It kinda does what I wanted in theory, but:

1. When you start and stop the game - it does not revert materials back to initial state, resulting in them referencing unloaded addressables (now null textures).
2. If you start the game again in this state - materials become pink and won't get fixed till you manually remove and re-add all references in spine assets
3. Or, in some cases, materials may start referencing hi-res original texture on game stop. In such state building addressables will generate duplicate of hi-res texture, defeating the purpose of having addressables in the first place.
4. Pressing Regenerate in AddressableTextureReference throws an error. It creates texture though.

[HaraldCsaszar]

@Gamer-XP Sorry to hear you're having troubles. Unfortunately we could not reproduce any of this behaviour, testing any Configurable Enter Play Mode options as well. Could you please send us a minimal Unity project which still shows this issue? You can send it as a zip package to contact@esotericsoftware.com, briefly mentioning this issue ticket URL so that we know the context.

Please also note that activating the placeholder textures via Testing - Assign Placeholders inside the Editor and entering and exiting play-mode is for in-Editor preview purposes only. Placeholder textures need not be assigned manually, as these are automatically assigned via a pre-build step (and reset in the post-build step).

[Gamer-XP]

Maybe I'm misunderstanding something. Actually, I planned on using those placeholders as temporary graphics behore Unity loads full-res assets from the server. So, I assume player may actually see them for a while if net or device is slow. That's why I find it weird that those placeholders are quite buggy.

And yes, after testing, this issue happens only if I used Assign Placeholders before starting the game - it won't revert materials back to placeholders after game end, instead I get null textures there.

Also, didn't know about pre-build step. Still, about that. I'm not 100% sure yet, but it doesn't seem to work with building addressables themselves at least. I tried doing analyze for duplicates, and seems like it still includes high-res texture in both addressable bundle and main project if you have spine object in the scene at least. This is the reason I tried to use low-res texture for the material as default - because I wanted to remove reference to the high-res one to prevent this duplication.

I'm using Unity 2022.3.30f, Runtime 4.2-beta btw

[HaraldCsaszar]

@Gamer-XP The placeholder assets are assigned automatically to avoid having blurry low-res textures during development, or having to replace them all manually before building, either of which would be terrible to work with.

That's why I find it weird that those placeholders are quite buggy.
And yes, after testing, this issue happens only if I used Assign Placeholders before starting the game - it won't revert materials back to placeholders after game end, instead I get null textures there.

If reset after exiting play mode does not work as expected, please send us a minimal reproduction Unity project as described above, so that we can fix the issue.

. I'm not 100% sure yet, but it doesn't seem to work with building addressables themselves at least. I tried doing analyze for duplicates, and seems like it still includes high-res texture in both addressable bundle and main project if you have spine object in the scene at least.

Did you assign the normal high-res textures at an addressable bundle? Or did you leave it at Addressable unchecked (which would be wrong)?

This is the reason I tried to use low-res texture for the material as default - because I wanted to remove reference to the high-res one to prevent this duplication.

Thanks for the info, sorry to hear. We would really like to fix the issue if it behaves like that in your project. As mentioned above, it would help us a lot if you could send us a minimal Unity reproduction project.

[HaraldCsaszar]

I'm using Unity 2022.3.30f, Runtime 4.2-beta btw

@Gamer-XP BTW: Unity version 2022.3.30f does not exist yet, 2022.3 is at 2022.3.11f currently. Did you mean 2021.3.30f?

[Gamer-XP]

Yep, sorry, a typo. 2021.3.30f.
I'll try making an example project later. A bit busy for now.

[HaraldCsaszar]

@Gamer-XP Great, thanks.

[Gamer-XP]

Here you are:
SpineAddressableIssues.zip
So, how to reproduce:

1. Windows -> Asset Management -> Addressables -> Analyze
2. Make suere there is full-res character in the scene
3. Press Analyze selected rules with all rules selected
4. Confirms there characters' texture is listed in Check Scene to Addressable Duplicate Dependencies - this means this texture is both in addressable bundle and in the main project now bacuse it's refrenced by the character's material
5. Replace texture in the character to the low-res version
6. Analyze again
7. No errors now, since low-res texture is not an addressable asset

[HaraldCsaszar]

@Gamer-XP Thanks for sending the reproduction project. Please note that the Addressables - Analyse functionality will report the original textures to be referenced twice by the scene and the addressable group, since the original full-res texture will be replaced as a pre-build step, which the Analyse run does not trigger. So thanks for bringing this to our attention, we will have a look if we can improve the situation to make the analyse run more useful and not report false-positives in this regard.

Regarding the other more important issues you reported above:
Unfortunately we could reproduce neither of the issues, everything worked without any issues using your project. Since your repro-project was created with version 2022.3.1f1, we used the latest 2022.3.11f1 version (note the .11 vs .1) for testing. Are you sure that you've been using the latest Spine On-Demand Loading Extensions and Spine Addressables Extensions UPM packages, version 4.1.0-preview.2?

[Gamer-XP]

Weird. It works fine in the example project for me too, but is bugged in the actual one. I'll try checking what's going on here.

[HaraldCsaszar]

@Gamer-XP Thanks for confirming. Please have a check whether the version in the original project is really 4.1.0-preview.2 at both packages, and not 4.1.0-preview.1 in the Unity Package Manager window.

[HaraldCsaszar]

Fixed an issue with editor not resetting on-demand loaded textures after exiting play mode, see commit 7eea8ce.

[Gamer-XP]

Thanks. I'll check it when I get time.
We've solved texture loading in different way for now (by swapping texture in materials, while textures are automatically generated when building addressables), and I didn't have time to check on-demand loading yet.

[liuxiaotian]

This commit looks like it increases the build time. With this commit included, our project's build time jumped from 10 minutes to 20. Reverting this commit could get our build time back to normal.

[HaraldCsaszar]

@liuxiaotian Thanks for reporting and sorry for the troubles. I assume you mean the last commit above, 7eea8ce, right?

We will have a look at it.

[HaraldCsaszar]

@liuxiaotian Unfortunately we could not reproduce the issue. When starting or ending a build, this callback of changing play mode should not be called at all. Even if it were called, only the assets of type OnDemandTextureLoader would be processed.

Which exact commit did you revert?
Which Unity Editor version are you using?
Could you please describe in more detail what your setup is? Are you having many Assets of type OnDemandTextureLoader in your project, or do you have any at all?

Could you add a Debug.Log() statement to GenericOnDemandTextureLoaderInspector.AssignTargetTexturesAtAllLoaders() which shows whether this method is called when starting or finishing a build in your project?

Or even better: could you perhaps create a minimal Unity project which still shows this issue? If so, you can send it to contact@esotericsoftware.com, briefly mentioning this issue ticket so that we know the context.

[liuxiaotian]

Sorry for the late reply. I'm using Unity 2020.3.41f1, the commit is 76e8538. SpineBuildProcessor.PreprocessBuild() takes around 70 seconds when executed directly, it takes longer during building. Here is my test code.

[MenuItem("Test/Spine Preprocess Build")]
static void Menu()
{
    var sw = new Stopwatch();
    sw.Start();
    SpineBuildProcessor.PreprocessBuild();
    sw.Stop();
    UnityEngine.Debug.LogError($"Total time: {sw.ElapsedMilliseconds}ms");
}

[sandolkakos]

hi @HaraldCsaszar, first of all thanks a lot for the initiative of On-demand and Addressables modules. We have a character with 70+ skins and all the textures is getting loaded into the Memory, but now with these modules we were able to solve this issue.

It has almost 1 month since we started to investigate how to use the modules, and came up with different results:

• ✅ Having the SkeletonGraphic already setup into scenes built-in the App (non-Addressable scenes) did work well.
  • This was our first prototype, so we were really happy that we found a solution, until we try it out in our real project...
• ❌ Having the SkeletonGraphic already setup into Addressable scenes still presented all placeholders and original textures loaded into Memory.
• ❌ Instantiating the SkeletonGraphic prefab using Addressables in the non-Addressable scene also did not work.
• ❌ Instantiating the SkeletonGraphic prefab using Addressables in the Addressable scene also did not work.

After days of investigation, today I found the reason for the 3 failures above...
The Materials were not set with the low resolution placeholders in the Addressable bundles.

By default, the Addressables Build process starts before the App Build process. That means this line below is executed only after the Addressables already got completely finished with all the Materials using the original High resolution textures.

• spine-runtimes/spine-unity/Assets/Spine/Editor/spine-unity/Editor/Utility/SpineBuildProcessor.cs

  Line 148 in 73318d6

  internal static void PreprocessOnDemandTextureLoaders () {

In our project, the build pipeline has already a system in place where I can trigger the
OnDemandTextureLoader.AssignPlaceholderTextures(); before the Addressables Build starts, but it would be nice if you could also add that automation to the plugin itself, so that people will not run into the same situation we were.

[HaraldCsaszar]

@sandolkakos Thanks very much for the feedback and for sharing your insights, these are very valid points! Sorry you were having troubles. We will have a look at what we can do to automatically cover these cases as well without adding any overhead.

[HaraldCsaszar]

@sandolkakos Thanks again for reporting your issue. The above commit for spine-unity (the core-package) on the 4.2-beta branch fixes pre-build-step order in Unity 2021.2, where Addressables and AsserBundles are built before the SpineBuildProcessor.PreprocessBuild hook is called.

Details: This was due to BuildPlayerProcessor subclasses callback hooks being called before IPreprocessBuildWithReport subclasses regardless of callbackOrder. Now SpineBuildPreprocessor is derived from BuildPlayerProcessor on Unity 2021.2+.

Since this could be a breaking change for existing build pipelines on 4.1, this change has been pushed to 4.2-beta only.
A new 4.2-beta spine-unity unitypackage is available for download:
https://esotericsoftware.com/spine-unity-download

The above change should automatically fix the order of pre-processing on-demand-loaded textures before building Addressables via Build Addressables on Player Build. Unfortunately we haven't yet found a way to automatically pre-process single AtlasAssets when building only Addressables via the Addressable Groups window. If anyone knows a solution to this, please do let us know!

[Gamer-XP]

For building addressable only, the solution I've made is custom shemes (AddressableAssetGroupSchema) + custom addressable builder script (BuildScriptPackedMode). I don't think there is any way to modify existing build pipeline for addressables. At least, I couldn't find anything.

[sandolkakos]

Thanks a lot for that fix, @HaraldCsaszar. I'm always happy to help.
We are not gonna update to 4.2 yet since we were able to handle that issue in our build pipeline, but I'm sure people will find it super useful and, in the future, when we update to 4.2 we will be able to remove our workaround. 👍🏽

[HaraldCsaszar]

@sandolkakos Glad it helps, thanks for your reply! If you find anything how we can further improve build automation, don't hesitate to let us know.

[sandolkakos]

In our project, we use Github actions to build the App, and before starting the build we can prepare a lot of stuff on the project.
For local builds in the developers machine, we have created a different script to interrupt the build when you click the Build button at Build Settings window in order to execute our preparations and then continue with the build process. Something like this:

[InitializeOnLoad]
public static class LocalBuildScript
{
    static LocalBuildScript()
    {
        if (Application.isBatchMode)
        {
            // This script should not be used in batch mode.
            return;
        }

        BuildPlayerWindow.RegisterBuildPlayerHandler(BuildPlayerConfirmation);
    }

    private static void BuildPlayerConfirmation(BuildPlayerOptions options)
    {
        // Implement the pre-build preparations here

        // Continue the build
        bool success = BuildPipeline.BuildPlayer(options);
    }
}

[sandolkakos]

But I'm pretty sure it will not solve the problem for build the Addressables via Addressables Group window.

[HaraldCsaszar]

@sandolkakos Thanks for the info, always much appreciated. Unfortunately I'm afraid this will cover just the scenarios that are already covered by the SpineBuildPreprocessor being called during builds by subclassing BuildPlayerProcessor or IPreprocessBuildWithReport. If I misunderstood anything or my assumtions are wrong, please don't hesitate to let me know!